【Chen Guo press】
The Japanese Software Industry: What Went Wrong and What Can We Learn from it?
It is an indisputable fact that Japan's IT industry, especially the software industry, lags behind Europe and the United States today. In fact, Japan's computer industry, which was developed from the electronic engineering industry, had a not low start - before the 1980s, when IBM dominated the computer industry, Japan was able to rival the United States in terms of hardware, both in large-scale computers and in the personal computer era. This also made Japan's semiconductor industry lead the world. However, the software industry has been lagging behind since the early 1990s. On the surface, the software education in Japanese universities lags behind that of the United States. The reason for this phenomenon is that Japanese companies emphasize hard over soft, and the IT industry does not pay enough attention to software productization. The root cause is the systemic problem of the economic system of Japan's large companies.
If we compare this industrial research article on Japan eight years ago with the current phenomena of China's software industry and the demand for application software by Chinese large enterprises, it can be said that it is really hard to beat. In addition to the fact that the foundation of the electronic engineering hardware industry in Chinese society is weaker than that of Japan, the phenomenon of the software industry is almost exactly the same as the criticized Japanese phenomenon:
- Large companies are spinning off their IT departments into technology companies
- Large enterprises are increasingly inclined to customize software development to meet their own needs, rather than using packaged software to promote their own changes
- Large enterprises increasingly use IT outsourcing, weakening their own IT capability building
- The enterprise itself cannot explain its IT needs clearly, and the service provider's ability to refine the needs is also very weak. A pair of confused people got together.
- Large companies do not respect the intellectual property rights of startups. After learning about the new ideas of startups, they wait for their old relationships to make the same thing.
- Senior leaders of the enterprise do not realize the transformative role and strategic significance of application software to the enterprise
Simply put, if productized enterprise software packages such as UFIDA and Kingdee and innovative enterprise service SaaS Companies cannot develop well in China. Large companies develop their own customized enterprise information systems and despise the intellectual property rights of software companies (for example, companies ask for the source code of manufacturers without principle, euphemistically calling it "independent and controllable"). Then software development will only be regarded by companies as a labor force (also known as "code farmers") service rather than a discipline. This will greatly hinder the technological innovation ability of China's software industry, and China's IT industry will repeat the mistakes of Japan's backward IT industry.
As far as Chinese companies themselves are concerned, if they do not pay attention to cultivating their own IT capabilities and their senior leaders do not realize that information technology is a strategic capability of the company, but only regard IT as a tool for companies to record information, then the so-called "digital transformation" will be empty talk.
This is also the advice of the original author and two Japanese and American scholars to policymakers in other countries besides Japan, and they are painstaking efforts to avoid the mistakes of the Japanese industry.
Japan’s software industry: What went wrong and how we can learn from it
About the author:
Robert E. Cole is Professor Emeritus of Business and Sociology at the University of California, Berkeley, and Co-Researcher at the Institute of Technology, Enterprise and Competitiveness (ITEC) at Doshisha University in Kyoto, Japan.
Yoshifumi Nakata is a professor of human resource management at Doshisha University Graduate School of Policy and Management and Doshisha Business School in Kyoto, Japan, and the director of the Institute of Technology, Enterprise and Competitiveness (ITEC).
Article summary:
Recent research results show that the Japanese IT industry is increasingly lagging behind the U.S. IT industry in software innovation, which is the fundamental reason why Japan's technological competition performance has weakened compared with the United States. This article explores various explanations for this result and analyzes the reasons for the trajectory of the Japanese software industry. These sources can be found in recent understandings of software's transformative role and its value-creating potential, as well as in the evolution of industry structures. Finally, the article considers how policymakers in other countries can learn from Japan's experience to build a more dynamic software industry.
(Keywords: high technology, software industry, information technology, path dependence, strategic management, Japan, learning)
It’s hard to imagine that just over 20 years ago, Americans were warned that Japan would become the next software superpower following its manufacturing success. Small, fragmented Silicon Valley companies are seen as unable to match the deep pockets of Japan's large, integrated high-tech manufacturers. Furthermore, the software development factory approach favored by large Japanese companies is seen as superior to the manual software workshop approach that dominates in the United States.
Despite these predictions, the past 20 years have not been kind to Japan's IT companies and high-tech manufacturing companies. In this article, we explore recent research on the impact of software innovation on national competitiveness, drawing on the experience of Japan, focusing primarily on the factors behind these phenomena, specifically why software-intensive products in the world's third-largest economy, in a country considered to be the most competitive manufacturing sector, have become largely less globally competitive. Corporate executives, government officials, and educational leaders in Japan severely underestimate the importance of software in competing across a broad range of industries and products in the consumer and corporate worlds.
This is a lesson for all economies that institutional leaders must not only sense and monitor but also act on emerging technological changes lest they themselves suffer. To be sure, it’s not easy to tell whether a possible technological change is a technological fad or minor development or an early sign of disruptive change. Additionally, it’s easy for agency leaders to get hung up on their existing vision for technologies that continue to generate positive revenue streams despite slowing trends. Japan's leaders, who were hugely successful at making hardware, found it much more difficult to envision software as an all-purpose companion, let alone a replacement model. Another plausible hypothesis is that the global weakness of U.S. manufacturing relative to Japanese companies gives U.S. companies stronger incentives to find, sense, monitor, and respond to new opportunities created by software. In the conclusion of this article, we will conduct a simple counterfactual analysis by hypothesizing what Japanese companies might have done differently to potentially reap relevant rewards during a period of increasing software importance. Our goal is to provide guidance to policymakers around the world seeking to build IT capabilities in their economies.
Japanese IT companies' share of major global consumer goods markets has been steadily declining. Not only have they failed to compete globally on their most transformative consumer products, such as smartphones and tablets, but they have also suffered strong domestic market disruptions with these products. Additionally, the functionality and features that come with these products, including mobile gaming and the rise of smart cameras, are impacting global companies that previously found success in console software and point-and-shoot cameras, and more recently appear to be eroding sales of sophisticated DSLR cameras.
From 2000 to 2011, Japan's output of domestically produced electronic products, most of which contain embedded software, fell by 50%. Exports of electronic products fell by 37% during the same period. Japanese electronics companies increasingly deal in intermediate products, producing components for other manufacturers. Consumer electronic components and equipment accounted for 45% of total production in 2000, rising to 56% in 2011, and during the same period, industrial components increased from 8% to 12%. The problem is that most of the profits from such products usually go to the sellers of the final product, such as Apple and Samsung; while component suppliers (other than those with strong proprietary technology, including system integration capabilities and/or strong intellectual property protection) are most likely to get some of the profits. As of the end of December 2011, the total market capitalization of Japanese electronics manufacturers had more than halved from 2000 levels. This decline reflects more than just a decline in market capitalization during this period; the TSE's first segment (large companies) fell only 16% over the same period. Even as Japanese tech companies maintain strong sales globally, their profit margins (already low compared to competitors) have fallen even lower in recent years, with many companies operating between 1-2% by 2009. This means less money is available for future capital investment relative to many foreign competitors. The extent to which the recently weakened yen can reverse this trend is an open question.
Recently, in order to avoid being hit hard in the consumer electronics field, Hitachi, Toshiba and Mitsubishi Electric have shifted their main business focus to large-scale social infrastructure projects (energy, transportation and utilities). For these projects, these companies view software more as the glue that holds different project components together rather than as the primary value creation factor. However, it is only a matter of time before some competitors with strong software capabilities, such as Siemens, establish a competitive advantage in these infrastructure markets by using software to drive front-end functions and services.
The IT industry is more than just electronic products. IT products belong to the information and communications technology (ICT) industry. According to the World Bank, services accounted for 5% of global ICT value in 1996, but this rose to 22% by 2009. Most of the value of ICT services is contributed by software. However, a troubling omen for Japan's future competitive performance is that services accounted for just under 2% of Japan's ICT exports in 2009. It's clear that Japan is lagging behind in the transition from hardware to services, and much of that weakness can be attributed to software weaknesses. Another factor contributing to their weakness in exporting ICT services is that product customization and system integration are often done in local languages based on knowledge of local business practices. It seems reasonable that Japanese companies are disadvantaged in exporting services because Japanese management's English skills are often weak.
Consistent with the above observations is the large number of Japanese software and information services imports, as well as the lack of groundbreaking software start-ups, and the virtual absence of a large and vibrant independent software industry. The small scale of software exports and reliance on high-level software imports are related to the weakness of Japan's domestic packaged software market, and the vast majority of applications are unique customized solutions (see Figure 1). In 2012, 86% of software sales in the Japanese information services market came from custom-developed software, while only 14% came from software products. It is worth noting that game software accounts for 37% of total software product sales and 5% of total software sales.
Still, Japan, the world's third-largest economy, is second only to the United States in software sales. Its embedded software capabilities are impressive. Machine tools, robots and automobiles are the three main locations for its high-quality, globally recognized embedded software products. Other studies rank Japan's software quality (measured as defects in various types of software projects) and the productivity of the software development process (code productivity) at levels significantly higher than a sample of U.S. projects. These are impressive achievements.
It is the gap between their strong process capabilities in software development and their weakness in product innovation, reflected in the lack of global presence in software products and services, that is at the heart of this analysis. Michael Cusumano (Note: Distinguished Professor at MIT Sloan School of Management) used the term "Japanese software puzzle" to describe this contradiction.
Recent patent research findings
In a recent article (“Going Soft: How the Rise of Software Based Innovation Led to the Decline of Japan’s IT Industry and the Resurgence of Silicon Valley,” The Review of Economics and Statistics, July 2013), Arora, Branstetter, and Drev reported the results of their study of U.S. and Japanese IT patents granted by the U.S. Patent Office (USPTO) from 1983 to 2004. The authors examine U.S. and Japanese non-hardware patents (actually embedded applications, systems, and enterprise application software patents) in high-tech areas. They found that while software patent citations were growing in both Japan and the United States, Japanese companies filed for fewer software patents than U.S. companies, and that this difference had grown steadily since the late 1980s, especially after the mid-1990s. Their findings suggest, first, that IT inventions have become increasingly dependent on software over time. Second, American companies are more aggressive than Japanese companies in incorporating software inventions into their products and services. Third, they found that U.S. firms' innovation performance improved relative to Japanese firms in the most software-intensive IT areas (computers and peripherals). Finally, the researchers found that within the IT industry, Japanese companies are now disproportionately located in less software-intensive industries.
They concluded that their findings provided a new explanation for the "sharp global decline" of Japan's IT industry in the 1990s. The authors discuss two possible reasons for the lack of software innovation in Japan. First, they pointed to the limited supply of software knowledge and skills (human resource constraints), and second, the slowness of Japanese IT companies in recognizing the industry-transforming nature of software. They concluded that the former, not the latter, was the main reason Japan was increasingly lagging behind the United States in software innovation.
University Computer Science Education
We build on their analysis and first add a piece to the puzzle by surveying U.S. and Japanese universities' efforts to equip a new generation of students with the knowledge, analytical skills, and tools to tackle the rapidly evolving field of computer science (CS). Although our focus is on computer science, our analysis also covers the broad field of information technology.
Given the importance of skilled human capital for software development, examination of IT software education allows us to assess IT human capital in the respective economies, in part, through the timing and quality of software investments. Steinmueller (Note: Steinmueller, professor of innovation at Stanford University and deputy director of the Economic Policy Institute) believes that the advantage of the U.S. software industry over Japan lies in its first-mover advantage, which is achieved by government R&D policies and early development of computer science education at the university level. The latter's advantage over Japan, and indeed over all other advanced industrial countries, continues to this day.
In the United States, about 20% of software developers have some graduate education, compared with 10% in Japan. The PhD gap is even wider than this 10% gap reveals. Until recently, most Japanese engineering PhDs were earned by industrial researchers at Japanese companies. These degrees (ronbun hakase, thesis doctorate, compared to katei hakase course doctorate) are usually awarded by the employee's previous university, usually after many years of research and development studies. No coursework is required, just submit a paper and publish some articles in reputable journals. These PhD holders are unlikely to be at the forefront of software innovation. Dissertation PhDs aside, most computer science PhDs (and most Japanese engineering PhDs) are obtained by those aiming for an academic career. Behind the limited number of Japanese PhDs in software-related private industry jobs is a chronic lack of overall industrial demand for engineering PhDs in Japan, as well as a dearth of software start-ups.
In contrast, most computer science PhDs from American universities are employed by industry or government. In the United States, engineers with PhDs and MScs have created and are creating substantial new value for private industry. From 1978 to 2008, the United States awarded approximately 22,000 computer science and information technology doctorates. Doctoral students from the best universities in the United States have experienced rigorous courses and internships. Many of them continue to invent, create and maintain a new world, such as UNIX and relational databases. Startups founded by people with PhDs include Adobe, Qualcomm, Google, Sun, Cadence, Synopsys, VMware and Symantec. There are also some PhDs, such as Edgar Codd (note: the inventor of database technology in the 1970s), who has a PhD in computer science and worked at IBM, but inspired startups such as Oracle to develop commercial products based on his pioneering research on relational databases. Doctoral contributions are Americans’ strength. It would be difficult to compile a comparable list of groundbreaking projects in Japan to those in the United States.
It is also worth noting that incorporating state-of-the-art software knowledge into Japanese science and engineering university curricula has been a long and slow process. In the 1990s, Japanese computer science professors realized that the computer science faculty in Japanese universities was weak. Until 2007, they relied heavily on the IEEE-CS (Institute of Electrical and Electronics Engineers-Computer Science) and ACM (Association for Computing Machinery) standards developed in the United States for successive course upgrades, but each time was delayed for a long time. Six years is a typical gap. Furthermore, even when these standards are adopted in the United States, they are often outdated. This lag suggests that Japanese educational institutions lack awareness of the speed at which IT technology is changing and the importance of adapting quickly to these changes.
The key to educating students on the latest curriculum is teachers with the latest knowledge. In this regard, Japanese universities are at a disadvantage. Much new information technology knowledge was developed in the United States, especially with the advent of the personal computer era in the early 1980s. At this time, Japanese IT companies, which had already achieved significant competitive success by imitating IBM's mainframe architecture, were still committed to building a new generation of larger mainframe computers. This misplaced focus has left them far behind the United States in developing, integrating, and deploying new software knowledge. Japan has a small number of startups and is more open to new knowledge, resulting in lost opportunities to close the gap.
The expansion of IT departments in Japanese universities in the 1980s was related to the shortage of trained computer science and information engineering teachers. To fill the vacancies, many universities have hired retired IT executives from large companies such as NEC, Fujitsu, Hitachi and Toshiba. These new scholars helped fill IT department faculty in the 1980s and 1990s. While they have real-world experience in IT business issues and some technical issues, most of their corporate training and experience is in mainframe hardware and software. In the United States, mainframes were quickly replaced by personal computers, workstations, newer programming languages, network systems, the Internet, and more.
Most Japanese teachers working in companies are not equipped to teach college students the most advanced software. At a time when research scientists (mainly from the United States) are transforming the field of computer science, most people with corporate backgrounds do not have the ability to contribute at the research frontier. All of this further pushed Japan and other countries further behind the United States in software innovation during the formative years of technology.
Professor Nobumasa Takahashi, a central figure in the Information Processing Society of Japan, analyzed the activities of national universities as they often set the direction for new technology courses for all other universities. He observed that the departmental structure of Japanese postwar universities was shaped by the continued expansion of fields such as civil engineering, mechanics, electrical engineering (EE), and chemistry. In many cases, new departments are formed through Ministry of Education budget allocations driven by these existing departments. “Therefore, the creation of new departments is strongly tinged with the expansion of old departments.”
The new Department of Informatics (jouhougaku), which is roughly equivalent to what Americans call information science, includes both the core of computer science and content from related scientific disciplines, among other restrictions. This resulted from the unexpected interaction of two developments. In 1991, the Ministry of Education de facto deregulated universities and no longer required science and engineering schools to offer liberal arts courses in the first two years of undergraduate education. Therefore, many science and engineering schools replace them with engineering and science courses. As a result, a large number of redundant teachers were created. Since they cannot be fired and they need to be transferred from their original departments, this dilemma finds an "accidental" solution.
Around the same time, the Ministry of Education imposed restrictions on the expansion of university departments out of concern about declining birth rates across the country. However, there are some exceptions for fields of growing national importance, and one such major is information technology. However, these universities took advantage of this exception to “offload” many of the redundant faculty mentioned earlier into these emerging fields. Many career-changing faculty continue to teach in their current majors but instead work in computer science and related departments.
Professor Masato Takeichi found that in 1998, the total number of faculty (2,615) in the Information Science Department of Nippon University of Science and Technology was roughly as follows: only 30% of the core computer science faculty whose academic majors were computer science and mathematical informatics accounted for, and an additional 45% included information science from a broader perspective, including systems informatics, electronic informatics, and intelligent informatics. However, it is worth noting that 25% of people were educated in a major that had little to do with computer science. So while the Ministry of Education recognized the importance of the new discipline to Japan's future and designated it as a growth area, they subsequently ignored the blatant destruction of the new field when filling new faculty positions. As a result, new departments were devalued by uncontributing computer science faculty. In turn, this means students have far less access to the latest computer science knowledge. More subtly, it makes it harder for computer science to be recognized as a distinct scientific discipline, but rather as a profession that even outsiders can fool around with.
A paper written by Baba, Takai, and Mizuta in the mid-1990s that focused primarily on mainframe software noted that they observed that computer science education was not well accepted in Japan and that most software workers acquired professional skills through on-the-job training. They further point out that, consistent with highly customized development, knowledge in the Japanese software industry tends to be accumulated internally and exploited primarily within the company, unlike the industrial drive in the United States. The Information Technology Promotion Agency (IPA) under Japan's Ministry of Economy, Trade and Industry (METI) recently warned that these characteristics remain major obstacles to software innovation to this day. In March 2013, Professor Fumihiko Kimura, a member of the Japan Science Council, stated at a public forum at the University of Tokyo that computer science (known as jouhoukagaku in engineering schools) is still not recognized as a clearly defined discipline in Japan by most stakeholders.
One measure of whether a new field becomes a recognized discipline is the willingness of those in the original established discipline to contribute to the emerging discipline. These contributions both reflected a positive assessment of the emerging discipline and gave further legitimacy to this new field. In the United States, for example, both theoretical and applied mathematicians played a key role in establishing the new discipline of computer science. Mathematical logic, Turing's and Gödel's theorems, Boolean algebra for circuit design, and algorithms for solving equations and other mathematical problems played an important role in the early development of computer science as a discipline. In contrast, Professor Masato Takeichi, who served as dean of the Graduate School of Information Science and Technology at the University of Tokyo from 2004 to 2007, reports that Japanese mathematicians are not as attracted to new IT fields as they are in the United States. He believes this is because computer science is not viewed as a distinct subject in Japan. Likewise, information engineering in engineering schools usually occupies a relatively low position in the hierarchy of engineering-related fields in Japan.
Overall, about 63,300 computer science and mathematics students earned bachelor's degrees in the United States in 2009, compared with only 16,300 in Japan. Considering that Japan's total labor force is about 43% of that of the United States and its nominal GDP is about 37.5% of that of the United States, this gap is much larger than imagined.
At America's leading engineering schools, majors like computer science are growing rapidly, attracting highly qualified applicants, with enough autonomy and strong incentives to grow their departments. Faculty will be incentivized to “put their stamp on” a new generation of bright young people, and they will use student growth to secure more departmental resources. The historically decentralized education system in the United States means that, unlike Japan’s education sector, the Ministry of Education has always played a non-actor role on these issues. As David Hodges, former dean of the School of Engineering at the University of California, Berkeley, said, "People tend to respect departments that show leadership." This reflects the entrepreneurial spirit of the top engineering schools in the United States to quickly adapt to new environments.
This adaptation process contrasts sharply with the practices of Japanese universities. We can take the example of the University of Tokyo, Japan's leading ICT department. As in all Japanese universities, in the Faculty of Engineering, there are prescribed student quotas for each undergraduate major (such as information and communication engineering). Before university deregulation in 2006, these quotas were strictly set by the Ministry of Education. Since the enactment of university deregulation in 2006, admissions restrictions have become the responsibility of faculty. However, the faculty in question are not those of a specific engineering department, but rather those of the engineering department as a whole. Therefore, when student quotas are discussed, each department tries to protect its own turf and not suffer at the expense of other departments. This is detrimental to emerging departments struggling to expand, and one consequence of this zero-sum game is that enrollment limits rarely change.
Taking the Department of Information and Communication Engineering of the University of Tokyo as an example, as of May 2012, the number quota for Electrical and Electronic Engineering (EE) students is 150 (junior and senior years combined), while the number of students in the Department of Information and Communication Technology is only 80. The distribution at Stanford University is quite different. The total number of undergraduates graduating in EE is 50 per year. This number has remained stable for some time, but in recent years the number of computer science degree graduates has been rising sharply. In 2012, there were 250 students graduating from computer science majors, five times that of EE majors.
The allocation of student quotas for EE and ICT majors at Japan's top universities seems more appropriate 20 years ago, when information technology and software were not such globally competitive forces. The current distribution model is suitable for traditional hardware-centered manufacturing education.
In summary, there is good evidence that Japanese educational institutions have been slow to incorporate strong software education into their curricula and still lag behind. This situation leads to human resource constraints in training software developers with up-to-date knowledge. However, these data are also consistent with the failure of Japanese businesses, governments, and educational institutions to recognize and act on the importance of software innovation. In the second half of this article, we suggest that these factors are, at best, direct causes of lagging software innovation. We will explore the position that they arise primarily from deep structural factors.
institutional and structural perspectives
A senior person in the Japanese software industry understands Japan’s software innovation issues as follows:
“Around 2000, we did have some labor shortages. The current problem isn't a labor shortage, it's that there aren't enough Japanese products focused on software, and there aren't enough Japanese software professionals focused on products. Both situations lead to a lack of demand for good software architects/designers. Overall, this leads to a lack of software innovation.”
In the following sections of this article, we will explain the factors associated with this perspective. Some people believe that human resources with specific skills are limited, and managers in the United States and Japan always complain that they lack sufficient opportunities to obtain truly top-level software talents. However, at the same time, the Japanese IT industry lags behind in software innovation, mainly due to weak incentives for companies to create innovative software products. If companies don't work hard to create such products, then they don't see the urgent need to hire, train, and use more and more highly qualified software people.
Lest we be accused of suppressing evidence of different perspectives, we note the following: In reviewing the academic and popular/practitioner literature on the Japanese IT industry over the past decade, we did not find widespread discussion that the Japanese industry has suffered or is suffering from severe software human resource constraints. In our interviews with leaders in Japan's software industry, government, and academia, no one told us that software industry growth or IT innovation are currently being constrained by severe labor shortages. The Information Technology Promotion Agency (IPA) annually surveys IT company executives on their perceptions of the adequacy of the IT labor supply. Those selecting "massive shortages" fell from 28% in 2007 to 5% in 2009, then gradually increased to 19% in 2013 (without distinguishing between the supply of high-skilled and low-skilled workers). The initial decline is likely due to the economic crisis of 2009 and the massive increase in outsourcing of enterprise and embedded software development to India and especially China over the past decade, primarily for low-level and mid-level programming jobs. The IPA data cover a much later period than that assessed by Arora, Branstetter, and Drev's Patent Study. Since the IPA's data does not show widespread belief among IT executives that there is a severe labor shortage, we can assume that such shortages by themselves will not largely explain Japan's IT industry's continued underperformance.
Our proposition is that the Japanese IT industry lags behind in software innovation mainly due to weak incentives for companies to create innovative software/IT products. Preliminary evidence for this assertion can be gleaned from comparative IT investment data provided by information technology research and advisory firm Gartner Inc. They report private sector IT investment as a percentage of annual revenue in 2007/2008. The figure for Japanese private companies in 2007 was 1.03. The comparable ratio in the United States in 2008 was 4.3%. If American companies spend 4 times more on IT annually than Japanese companies, and if they allocate it efficiently, they are likely to produce more software innovation than their Japanese counterparts.
Gartner's data is further broken down into eight industries. The biggest gap is in the processing and assembly manufacturing industry. U.S. companies' investment in IT accounted for 3.5% and 4.6% of annual revenue respectively, while Japanese companies' investment accounted for 0.69% and 0.75% of annual revenue respectively. In only one industry did Japanese private companies outspend U.S. companies—the banking/insurance/securities industry, at 5.89 in Japan and 5.1 in the United States. The higher investment by Japanese banks in this area may be due to Japanese banks' greater reliance on mainframes (whose software upgrades are particularly costly), while U.S. banks have moved more to client-server systems.
Perhaps more important than differences in the scale of IT investment are differences in the deployment of IT investment across industries (as Gartner's survey results indicate) - the type of IT investment (Japanese companies invest more in custom development than producing software products, and have a stronger strategic focus on hardware than software) and the purpose of IT investment (Japanese companies are more likely to custom develop software to support their own current business practices, rather than taking advantage of new opportunities provided by innovative IT technology to promote enterprise change and transformation). These differences are further elaborated below.
We first examine this question by examining the contribution of software to total factor productivity (TFP) at the country level. Kazuyuki Motohashi (Note: Professor of Economics and Innovation Management at the University of Tokyo) conducted such an analysis of the contribution of software to TFP within three selected time periods. In the United States, software's contribution to total factor productivity has been increasing, from 3% from 1960 to 1995, to 6% from 1995 to 2000, and to 8% from 2000 to 2006. During the same period, software's contribution to Japan's TFP was almost non-existent: -3%, -1% and 1%. The differences between the two economies are consistent with our interpretation that there is a misallocation of IT investment in Japan relative to the United States.
Second, we delve deeper into these issues by examining the components of IT investments. Table 1 compares the allocation of software investments by investment type in the United States and Japan. Japan's data for 2000 show a heavy reliance on outsourcing (70%), which seems contrary to the literature that emphasizes the low level of outsourcing by Japanese companies. However, outsourcing in Japan, which focuses on customized development, is mainly the result of participation in the software development process of Japanese domestic companies.
As shown in Table 1, only 20% of Japanese companies develop internal software for their own use. These two types of custom development account for about 90% of Japanese software investment, while packaged software only accounts for 10%. This is roughly consistent with the sales distribution of custom development software cited in Figure 1. Longtime software analyst Nobuyuki Yajima wrote in 2012 that he believed Japan's data had not changed much since 2000, although he estimated that packaged software usage in recent years was closer to 15%; in the United States, it was 29%.
Comparable data for the United States in 2010 contrast sharply with Japan's distribution. They show the relative balance between investments in packaged software, outsourcing, and in-house software for a company's own use. In terms of internal investment in self-use software development, the US private industry is twice as high as Japan (37% vs. 20%). These differences are consistent with the weakening of IT capabilities at Japan's large manufacturing and service companies in the 1990s, when companies spun off their IT departments into subsidiaries or relied more heavily on systems integrators. In some cases, the motivation is to exploit new technology or market opportunities, but companies also use these spin-offs to reduce labor costs. At that time, Japanese high-tech companies were still relatively competitive globally, and they did not believe that their success depended on IT. Therefore, they believe that an easy way to reduce costs is to reduce the enterprise's IT staff.
The differences mentioned above are reflected in the distribution of IT employees by company type. The IPA reports that approximately 75% of Japanese IT technical employees are located in IT service industry companies (large system integrators and their layers of subcontracting companies, often characterized as "software factories"). In most of these companies, the opportunities to create innovative software products and sell them to large numbers of customers are very limited. A software vendor's first priority is to meet the cost, delivery, and quality specifications of a specific customer. In this environment, suppliers have little incentive to provide solutions with the latest system architecture, and product innovation is generally a low priority. It is not uncommon for projects led by top-tier suppliers to be actually executed by multiple layers of subcontractors, with the general contractor rarely involved in the specific technical execution.
The IPA also reports that the distribution of U.S. IT employees is almost the opposite of Japan's, with only 29% of U.S. IT technology employees located in IT services industry companies. Even allowing for the considerable underestimation of IT services industry employees on the U.S. side, the respective distributions in the report do illustrate the vast differences in software talent by industry location. They also suggest that U.S. software workers are better equipped to innovate software. When it comes to software innovation, the industry location of the talent matters.
Yasuhide Hosokawa offers an alternative, more modest explanation for Japan's lagging IT investment and its associated consequences. Hosokawa is the former executive director of the Japan Users Association of Information Systems (JUAS), an organization of more than 1,000 of Japan's leading businesses and IT users, and has been an influential voice in government IT policymaking. He expressed little concern about the slump in IT investment in Japan for four reasons: First, he cited data showing that the unit cost of a software developer in Japan is about half that of the United States. Secondly, he pointed out that Japanese enterprise application systems have a long lifespan. According to data from the Ministry of Economy, Trade and Industry, the average lifespan is 17 years. He acknowledges that the average lifespan of U.S. enterprise application systems is much shorter, but he believes this may be due to U.S. chief information officers (CIOs) constantly coming to power in an attempt to achieve career success by adopting new application systems, resulting in unnecessary waste of IT investment. Third, he pointed out that over the past 15 years, Japanese companies have significantly cut costs by moving information systems functions to subsidiaries with lower wage costs. Fourth, he presented data showing that Japanese IT suppliers are providing high-quality and high-productivity services at a lower cost than their U.S. and other competitors, with Japanese companies being the main beneficiaries.
What can we say about the interpretation of this dissenting set of opinions? What his four explanations reveal is that Japanese companies have chosen to compete on the operational efficiency (cost, efficiency and quality) of the IT industry rather than seek new revenue growth through innovation. This is a traditional Japanese hardware competition model that many Japanese manufacturing companies have effectively adopted in the past. This competitive model can work in slow-moving technology areas before competitors catch up. In rapidly evolving technology fields, such as IT, the limitations are obvious. In these areas, customers are often eager to pay for high value-added innovations, and companies compete strategically. Furthermore, consistent with our previous discussion, one of the consequences of emphasizing operational efficiency when developing enterprise application systems is a heavy reliance on generally trained IT personnel rather than a substantial need for highly trained software professionals.
Hosokawa admitted that the average lifespan of Japanese enterprise application systems is 17 years, which may be a limitation of enterprise application systems competing in operational efficiency. This means that, in practice, Japanese companies essentially abandoned many IT innovations over the course of 17 years in order to control IT investment costs. The extreme customization development associated with adopting new enterprise application software systems is expensive and therefore not easily updated and replaced. In making such decisions, Japanese IT decision-makers deprive their companies of the opportunity to use new IT technologies to innovate in a timely manner—to improve their business decisions, to provide further benefits to existing customers, and to expand their own business. It would be an exaggeration to attribute the rapid uptake of new technology by corporate America to new CIOs making unnecessary IT investments to enhance their career success. In short, we see the rationale behind the current Japanese model, but it seems destined to leave Japan further behind in software innovation.
In this context, it is crucial to identify the different impacts of software startups in the two economies. For truly innovative software, large U.S. tech companies rely heavily on software startups, which attract the most creative talent. Large public companies are often unwilling to tolerate the significant costs of learning failures associated with developing breakthrough software technologies. Instead, U.S. venture capital financed these risky new technologies. In the United States, software companies receive a significant portion of total venture capital investment.
In 1989, 0.04% of total venture capital investment in Japan went to software startups; by comparison, 11% in the United States at that time. As of 2011, total venture capital investment in the United States was approximately $29 billion, while Japanese investment accounted for only 12% of this total. At this time, investments in computer software startups account for about 24% of U.S. venture capital-funded startups, and more broadly, IT investments account for 57% of all U.S. venture capital investments. For Japan, software startups account for only 9% of venture capital startups. So not only is the proportion of venture capital investment devoted to software development much lower in Japan, but the total venture capital investment is also much lower.
Given the relatively low cost of starting a software startup, one might assume that software startups would make up a higher share of all venture-backed startups. To test this possibility, we used data from Teikoku Data Bank (Japan's largest credit reference agency) on all annual startups, not just venture capital-funded startups. Of the 5,292 companies that met the inclusion criteria from 2005 to 2011, only 7% were software companies. Therefore, the much smaller number of software startups in Japan is likely to both contribute to Japan's weak software innovation capabilities and deprive Japan's large technology companies of potential products.
The former CEO of Indian outsourcing giant Infosys Technologies estimated in 2010 that nearly 80% of annual IT spending in Japan was spent on maintaining and operating existing systems, compared with no more than 60% in the United States. Even allowing for some exaggeration, the result is that Japan's IT budgets for software projects that fund innovation are far smaller than the corresponding IT budgets of U.S. companies. These observations apply to an enterprise's management information systems (MIS) department's application software budgets, as well as to the budgets of embedded software and specialists that typically belong to product development departments.
As shown in Figure 1, system integrators account for 64% of total custom development software development sales in Japan. Therefore, it is important to understand the system integrator's relationship with the company's customers. Enterprise customers are often unable to capture and express their own IT needs well because customers have weak IT capabilities, which is caused by their own weak personnel capabilities. In addition, many people on the corporate side do not understand their own weak capabilities. This makes it difficult for system integrators to understand and respond to their needs, which can lead to a mismatch between delivery and requirements. For example, a customer might say they want a systems integrator to help reduce inventory. However, that may not be their real concern; what they may really care about is trying to understand the risks associated with different inventory levels and how to hedge those risks.
The customer's IT department may be of little help in clarifying customer needs because they are primarily responsible for funding IT systems and have little knowledge of the real business activities of other departments. However, system integrators must often act on the opinions of their customers rather than against them, as a "customer is king" mentality pervades Japanese organizational culture. Because enterprise customers often don't understand their own capabilities gaps, they may need specific business solutions but not know the full consequences of implementing those solutions. On the other hand, system integrators can make money by complying with customer requirements rather than challenging customers with other innovative and transformative solutions. At the same time, top managers also understand to some extent that they are short on IT talent, which makes them often risk-averse when faced with new strategic IT opportunities.
The high dependence of large companies on their systems integrators, and the lack of awareness of the strategic importance of IT in many companies, can be seen in the low proportion of large Japanese companies with full-time chief information officers (CIOs). Software experts at Mitsubishi Research Institute estimate that in 2008 only 30-40% of large Japanese companies had full-time chief information officers. In 2008, few U.S. companies with annual revenue exceeding $500 million did not have a full-time chief information officer. A latest survey by the Japan Users Association of Information Systems shows that more than 50% of large Japanese companies have chief information officers, but they spend only a little more than 10% of their time on IT work.
Of course, simply having CIOs work full-time on IT does not guarantee that they will play a strategic role for the company. However, the absence of the CIO role suggests that large Japanese companies are making far less effort in this area, and the CIO job itself is considered non-strategic. This situation reflects the long-term view of top managers in Japanese companies that IT is a cost center rather than a strategic activity that can promote the company's development. Most importantly, this is the root cause of Japan's weakness in software innovation relative to the United States. Any difference in the supply of IT human resources is an institutionalized misallocation of software professionals. As mentioned earlier, the vast majority of Japanese software professionals focus on customized development and the exigencies of their needs, rather than on creating products and services targeted at numerous enterprise customers in horizontal markets.
Another measure of the lack of a product-oriented IT strategy is the relatively slow advancement of agile (iterative) software development by Japanese companies compared to its widespread use in U.S. companies. Agile development is particularly valuable because of its ability to meet dynamic customer needs through ongoing customer/user collaboration and feedback. The companies mentioned above have a lack of IT capabilities that prevents them from adopting agile development methods. While companies increasingly push systems integrators to adopt agile, system integrators are often reluctant because their large corporate customers lack the IT capabilities.
Japanese companies have also been slow to create and use state-of-the-art enterprise software to create agile enterprises that can respond quickly to strategic opportunities. We see this in their slow and limited adoption of enterprise resource planning (ERP). Even when implementing ERP, Japanese companies tend to select only a few modules, typically personnel management and/or financial accounting modules. They also tend to implement only local business optimizations rather than enterprise-wide optimizations. Therefore, the results of their ERP implementation are often only the optimization of subsystems and partial systems for the enterprise.
Since enterprise software systems integrate the internal and external information management of the entire organization, in principle, they make company management not only more effective, but also more strategic. Past surveys have shown that U.S. companies are more likely to use IT investments for strategic benefits, such as winning new customers, increasing sales, and obtaining market information faster. In contrast, Japanese companies are more focused on obtaining operating income. This was further confirmed in a 2013 survey conducted jointly by the Japan Electronics and Information Technology Industries Association (JEITA) and the Japanese subsidiary of the US consulting firm IDC. They conducted an online survey of company heads and senior managers (excluding IT managers) at 216 U.S. and 196 Japanese multinational companies. When asked about their expectations for IT investment, Japanese companies’ first choice is to use IT to improve operational efficiency/reduce costs (48%), followed by using IT to enhance the development of products and services (22%). The first choice of American companies is to use IT to enhance the development of products and services (41%), followed by using IT to transform business models (29%). Business model change ranks seventh among the choices of Japanese companies, with only 13% of companies choosing to change. Consistent with these results, 75% of U.S. companies believe IT investment is very important, another 20% believe IT investment is important, and only 2% responded "can't tell." In contrast, Japanese responses were 16%, 53% and 24% respectively. It can be seen from these choices that Japanese companies are unlikely to view IT investments as strategic investments, which can only reduce their demand for high-level software talents.
Path dependence and hardware centrality
What explains the lag in recognition of the importance of software among Japanese business, government and education leaders? Here we add a piece of the puzzle: manufacturing's history of hardware-centric success and its impact on how companies approach embedded software.
We draw on the seminal contributions of Arthur Stinchcombe. He pointed out that an organization established at a specific time and place must make use of available social resources to construct its social system. This, of course, includes human capital, which is considered necessary to enable an organization to grow and survive. His particular insight was that an organization of a particular type, once established, tends to retain many of the basic characteristics it had when it was founded. Key organizational characteristics are imprinted so that the incentives that drive performance and the traits of the people initially recruited are often persistent.
Stinchcombe emphasizes the importance of understanding the persistence of these fundamental characteristics over long periods of time—the path dependence of organizational and technical learning that researchers propose. A company's previous organizational routines, norms, and corporate culture (its history) tend to resurface over long periods of time, constraining its future behavior. This is partly because the opportunity and motivation to learn and grow are often close to previous abilities. The more a company uses a technology, the better it becomes at it, and the higher its cumulative sunk costs in human and physical capital become. If a company uses a technology over the long term, growing returns may solidify its commitment to the technology.
One can apply this understanding to the hardware-centric origins and continued commitment to technology of Japan's legacy electronics and other high-tech companies. Their success strengthened these capabilities over a long period of time. Take electronics companies as an example. Electronic engineers dominate the founding process of these companies. As these companies evolved into IT companies, this professional expertise became dominant among management. It seems reasonable that this history continues to limit their transition to software.
As mentioned above, skilled human capital remains the most critical factor in software innovation. The quantity, quality and deployment of this skilled human capital are decisive. Sony has always been one of Japan's quintessential hardware-centric companies. The Walkman epitomized Sony's technical capabilities, a triumph of electronic engineering. Since then, a major part of Sony's innovation has been to make its products smaller, thinner, and lighter—mostly through creative hardware design. However, they have been weak in developing innovative application software. Former Sony president Howard Stringer revealed a major source of this weakness in a 2006 interview. The following paragraph specifically talks about the issue of human capital allocation.
“In the beginning, we did not involve software engineers in product development. Hardware engineers will start developing the product, and then software engineers will get involved. This is because in a job-for-life company, older people are at the top and younger software engineers are at the bottom, working their way up. So there is a generation gap.”
Stringer explains why software engineers enter the product development process late in Japan's age- and seniority-based promotion system. However, it's not just that older hardware engineers have more seniority. Further exploration of Stinchcombe's analysis can shed some light on these other factors. These managers are now in the upper echelons of leadership, having risen to their positions at a time when hardware capabilities became a critical factor in their personal and company success. In this case, senior executives tend to associate success with hardware innovation, unconsciously underestimating the growing contribution that software and service orientation can make.
Research has long shown that managers are most willing to promote people who are most like them. Managers believe that people who are similar to themselves are easier to communicate with, understand, and trust. In a recent study of hiring at professional services firms, Lauren Rivera, a professor at Northwestern University's Kellogg School of Management, found that evaluators used their experience as a selection criterion. Because the evaluators have had fairly successful careers, they believe that candidates with similar work experiences also have a higher likelihood of career success. Therefore, if a hardware engineer occupies a top management position, they will likely feel most comfortable promoting a hardware engineer. They know what hardware engineers are doing and can plug in, but software engineers produce intangible products that are difficult to understand. These factors have slowed the inevitable transition to the software age.
Leadership's lag in recognizing current realities is not unique to Japan, but may be more widespread, as the age and length of the service system delays the promotion of younger employees with new capabilities. The company has had extraordinary success in the past with hardware manufacturing, which has forged strong ties to the ideology of Japanese corporate management. As the older generation of leaders leaves, changes are taking place. However, it is questionable whether the pace of change in Japanese companies can keep up with global competition.
In the late 1990s, Japanese IT companies faced the hollowing out of manufacturing capabilities, lost key global markets, and were challenged by modular production and Asian competitors. It is in this challenging and uncertain environment that the traditional Japanese concept of "craftsman spirit" (monozukuri) is brought up. In their classic study, Barry Staw and his colleagues concluded that when individuals or organizations are threatened, they tend to become less adaptive. Instead of innovating, they tend to ossify their thinking and actions and revert to previously known behaviors.
Craftsmanship is a commonly used term in Japan to express the unique spirit of Japanese workers when making things. Japanese critics often call Japan "the country of craftsmanship." The term was revived, spread, and came to characterize the core competencies of postwar Japanese manufacturing. It refers to Japan's special capabilities and practices in manufacturing high-quality, constantly improving, and pursuing perfection in precision hardware products. The spirit of craftsmanship was elevated to management dogma and national strategy.
In 1999, Japan promulgated the Basic Law for the Revitalization of Core Technologies of the Craftsman Spirit. In the bill, craftsmanship core technologies are defined as technologies related to the design, manufacture, or repair of industrial products, and are further designated by cabinet order as general technologies that support the development of manufacturing industries. First, target areas of support include improving the supply and quality of skilled manufacturing workers, “who are the main supporters of core technologies of craftsmanship.” The new bill also targets research and development, promoting the development of industrial agglomerations, start-ups, small and medium-sized enterprises, and promoting the learning of craftsmanship. As a result of the new law, various government agencies began launching activities related to craftsmanship in a quest to restore corporate health and Japan's prosperity. Companies themselves have begun using the term to guide some of their initiatives. Every year, there is an Craftsman Spirit White Paper (translated as "Manufacturing White Paper"), which reports on the current status of the industry, challenges, and corporate and government responses. Above all, craftsmanship, as promoted and understood by manufacturing managers, is about maintaining and building Japan's hardware capabilities. Discussions of craftsmanship rarely involve software. When this happens, software is often viewed simply as an enabler, assistant, and controller of hardware capabilities rather than a driver of customer value. Even if its potential to create value is acknowledged, such as the 307-page "2012 Craftsman Spirit White Paper" released by Japan's Ministry of Economy, Trade and Industry in 2013, only 4 pages are about software (pages 95-98). Surprisingly, this situation still exists in 2013.
Related to this is the aforementioned trend of large, established Japanese companies entering new areas by creating new divisions and spin-offs. They tend to staff new divisions and spin-offs with employees from the parent company, who are likely to carry over existing work routines and management practices. This limits the ability of these new entrants to innovate. Furthermore, due to their dominant purchasing power, established companies and their choice of entry areas tend to stifle opportunities for true startups. Measured by sales, no new competitor has emerged among Japan's top electronics manufacturers in more than 50 years.
In the United States, more established electronics companies are disappearing and being replaced by new companies. These new companies may have no commercial or emotional stake in the old technology and associated work routines and management practices. Among the 21 largest electronics manufacturing companies in the United States in 2010, 8 did not exist in 1970, and 6 were too small to be among the Fortune 500 10 years ago.
All things being equal, large Japanese electronics companies and the new circuits they support are more likely to perpetuate their organizational culture into new areas and slower to adopt fashionable practices and routines. Consistent with this overall pattern, it’s not surprising that they have been slow to recognize and adapt to the growing importance and benefits of cutting-edge software. In the United States, by contrast, evangelists of new concepts where software matters tend to be independent start-ups. These new companies embody state-of-the-art practices, norms, and conventions, and their commercial success highlights the power of their products and management practices, driving existing companies to adopt their IT innovations.
Mitsuhiro Fukuzawa (Professor of Management, Seikei University) traces Ricoh Co., Ltd.'s long and painful journey from 1987 when it launched its first digital multifunction printer centered around copy hardware, at a time when software was largely viewed as trivial. After six generations of product architecture and 150 new models, they finally produced a sophisticated digital multifunction printer (MFP) in 2001. It is the first product to have a software project leader, software design leads the product architecture (its GW architecture), and product design is based on strong collaborative relationships between hardware and software engineers, as well as between copying and other functional departments. Adapting the hardware capabilities to produce a satisfactory digital MFP did not appear until the early 1990s, and we can say that this learning process took at least 8 years.
The product was produced in 2001, using the GW software architecture, and achieved commercial success in Japan. In fact, since 2001, many digital MFP products, including Ricoh's main products (such as mid-range MFPs), have successfully used the GW platform. Ricoh's Japanese competitors, such as Canon, have been no faster or more successful than Ricoh in developing high-performance digital MFP solutions. If not, Ricoh will suffer significant market losses on these products. However, it is difficult to see from an outside perspective that as a timely solution, a large and complex manufacturing company spent 8 years starting in 1994 to recognize that software design had become the defining fact of the entire product architecture and to take action. This article cannot capture the length and complexity of Ricoh's efforts to bring together all the organizational systems and functions needed to solve the problem.
Why did it take so long? Putting all the threads of our analysis together, a reasonable scenario looks like this: At Japanese companies like Ricoh, Canon, and Toshiba in the copier/printer business, the hierarchy of decision-making and status has always been mechanical or chemical engineers at the top, then electronic engineers, and software engineers at the bottom. Nonetheless, across a wide range of manufacturing industries, software engineers increasingly constitute the majority of the software development costs for embedded software products. In the 2010 annual survey of embedded products by the Japanese Ministry of Economy, Trade and Industry (METI), software development costs accounted for 61% of the total project development costs, and hardware development costs accounted for 24%. Even in this case, and even though software engineers make up the majority of the product development workforce, intense political dynamics are at work to maintain status quo power hierarchies that keep software engineers away from key decision-making roles. As a result, hardware-centric design tends to perpetuate, and the role of software is often seen as an assistant and controller of hardware functionality.
Until the market convincingly proves that these solutions no longer work, the mindset will change and the political power of software engineers will begin to rise. One reason for Ricoh's lengthy learning curve is that there aren't a large number of new graduates in computer science and related fields with the latest software skills entering companies willing to take advantage of their talents. Compared to the United States, Japan has relatively few young people starting companies without conformity. Startups can more easily integrate software into new product development, and they can serve as role models for those existing companies.
If a company frequently brings in software people late in the product development process, it can lead to a self-fulfilling prophecy that software doesn't contribute much—it's basically just a set of tools that enable hardware-centric product design. In a recent study of a complex automotive product in Japan that required the coordination of mechanical, electronic, and software engineers, Fujimoto and Park found that the status and power of embedded software engineers in Japan was low. When embedded software engineers are called upon to perform their tasks, the product development process often falls behind schedule and the software engineers are under intense pressure to complete their tasks. In this environment, software engineers are unlikely to fully understand and take advantage of hardware design to make innovative contributions. In short, the price of hardware priority is the sacrifice of software innovation optimization.
Outsiders often comment on the high quality of Japanese embedded software, believing that embedded software is a type of software with strong innovative capabilities among Japanese manufacturing companies. Here, they confuse quality and innovation. Japanese manufacturing companies do prioritize product quality in both hardware and software. They apply rigorous hardware quality methods to software quality. This has led them to focus on eliminating software bugs, even in non-mission-critical systems. However, these practices may not always be appropriate for software, and in any case, they have little to do with stimulating innovation and may even interfere with it. They also slow down market entry. To enable embedded software engineers to develop innovative solutions, they need to be involved early in the product development process, be able to collaborate fully and equally with hardware engineers, and be given the resources and time they need to develop their capabilities. The problems described by Stringer, Fukuzawa, Fujimoto and Park are also reflected in many other Japanese high-tech companies.
In our field research, further examples demonstrate how the deployment of software personnel is related to positions of power and control. A Samsung manager described the consequences of the company developing a hit product. A few years after the product was released, it was discovered that the hardware engineers working on the product were promoted faster than the software engineers working on the product. An internal study of why this was happening was quickly launched, and it revealed that the top managers making promotion decisions were hardware people. The contribution of software engineers is not as obvious to them as that of hardware engineers, so software engineers are promoted more slowly.
In short, the history, culture, and power of an organization determine how performance metrics are applied, and those involved are often unaware of the biases these metrics exhibit. The actions of Samsung executives demonstrate their priorities. They demonstrate leadership by first noting that they are aware of the situation, second by defining it as a problem, third by trying to understand why the situation is happening, and fourth by developing countermeasures. Japanese companies could benefit from this proactive effort by Samsung executives to overcome the inertial organizational forces that emerge in the software/services transformation.
Conclusion
In summary, we found considerable evidence to support the argument that there are not enough Japanese products focused on software, nor are there enough software professionals focused on products. Both situations create a lack of demand for good software architects/designers. The end result is insufficient software innovation. Finally, we also see that the independent software industry, which is largely composed of startups, does not play a transformational role in the Japanese economy in the same way as it does in the United States.
As we look back at how the Japanese software industry got to where it is today, it is valuable to identify specific inflection points where decisions by a different set of managers, governments, or university administrators might have led to a more positive trajectory for the industry. Policymakers in countries other than Japan would likely benefit from such analysis.
Here are some instructive scenarios where we think policymakers could have put the Japanese software industry on a more positive path. Some of these may be more realistic than others for Japanese policymakers. However, even those unlikely policy options, such as emphasizing strong English language skills, may be more realistic policy options for policymakers in other countries seeking to build a vibrant IT industry.
1. The Japanese software industry has created a large number of software "blue-collar" jobs in the "software factories" of the IT service industry for those with ordinary software skills. This is a huge factor hindering software innovation. As mentioned earlier, the driving forces in this software factory environment are delivery time, quality and cost targets, which are achieved by the standardization of processes, tools and module reuse. This result, while having important benefits, is due to the company's failure to properly recognize the differences between the software industry and manufacturing. Could Japanese companies have recognized earlier the limitations of this approach and the constraints it imposes on developing the innovative potential of software? The currently dominant software factory model reflects the cumulative impact of two factors: first, excessive outsourcing, and second, excessive customized development.
With the exception of Germany, most countries seeking to build viable software industries today do not have manufacturing as advanced as Japan's, so there is less pressure to imitate successful practices in hardware and develop custom software. However, there are still challenges in measuring how much outsourcing is appropriate. We can identify a possible inflection point in the early 1990s that may have formed a strong trend of domestic outsourcing and shrinkage of IT capabilities by large Japanese companies. As mentioned above, in the early 1990s, the IT capabilities of Japan's large manufacturing and service companies shrank. Large companies split their IT departments into subsidiaries and began to rely more on these subsidiaries and other systems integrators and their subcontractors. The common motivation behind these initiatives is to reduce labor costs.
If companies, instead of responding to strong pressure to reduce overall costs, recognize the value creation potential of IT, IT will evolve into one of the core competencies of modern companies. It seems reasonable that if large Japanese companies decide to retain more in-house capabilities, these companies will be better able to identify new IT opportunities and develop in-house software that more closely meets the needs of internal and external customers. Even today, there is a general consensus in the Japanese IT industry that most Japanese top managers still do not understand the value creation potential of IT and its competitive advantages. With stronger internal IT capabilities, companies will have a stronger internal voice to inform, educate and lobby top management on IT policy.
2. Japan will also benefit from the establishment of courses related to software transformation capabilities in elite educational institutions from which senior managers are selected. At the very least, when engineering schools are no longer required to teach liberal arts courses in their first two years, these schools can and should choose to establish strong required courses that address emerging global science and technology trends. This will definitely increase the understanding among young techies of the importance and breadth of the IT revolution. In the current global environment, all things being equal, companies with strong internal IT capabilities should have better results for the economy as a whole. The lesson here, therefore, for those policymakers looking to grow the IT industry is the importance of educating top institutional leaders across all sectors about IT’s value-creating capabilities. Even if they don't plan to develop their own software, companies need the ability to monitor and assess the rapidly evolving global IT environment. Similarly, executives in all departments need the ability to rationally interact with IT experts to make strategic IT procurement and deployment decisions.
3. We’ve written about the weaknesses of Japan’s independent software industry, specifically the shy nature of software startups. It's also part of an overall weakness in Japan's startup industry. Over the past 15 years, the Japanese government has been stimulating entrepreneurship through various measures. These measures, which include better tax treatment for new businesses, reducing start-up costs and bureaucratic procedures when creating new companies, providing knowledge to start-ups, establishing regional clusters, and improving the ease of transfer of university innovation to private industry, have not had the desired effect individually or in combination.
In contrast, little effort has been made to identify and remove institutional barriers. Related to this is the already well-discussed practice of large, established Japanese companies entering new areas through the creation of new divisions and spin-offs. They tend to provide clear and conspicuous direction, if not control, of new divisions and branches, which they staff with parent company employees who may continue the parent company's existing work routines and management practices. This tends to limit their ability to innovate; most importantly, corporate spin-offs are unlikely to pose a challenge to the existing parent company.
Related to this is the "keiretsu" vertical integration relationship established between the parent company and its suppliers. Scholars in both Japan and the United States have written extensively about these relationships, which embody what are known as “relational contracts.” These relationships include long-term market exchange relationships with product and service providers and are based on mutual trust, inter-firm learning and reciprocity. If a new startup comes up with an innovative solution (product or service) and tries to sell it to an established company, the established company is likely to turn to its existing suppliers and encourage or even support them to find a comparable solution. They will wait for a solution from their old relationship rather than simply switch to that new startup. These practices are particularly prevalent in the enterprise software market.
Working with startups is often a high-risk business. Their management is untested and they may not be able to deliver reliably, let alone be able to deliver on their promises. They are not yet considered compliant players. In this environment, corporate clients are often reluctant to conduct due diligence on a potential new partner. A related issue is the lack of exit opportunities for investors given the softness of Japan's domestic M&A market. Finally, there are other institutional barriers that exist in Japan's patchwork of national and local government regulations that serve to deter new entrants.
In the past, Japanese companies' restive strategy of using spinoffs seemed to work well. However, when faced with the strong challenges posed by first movers in rapidly developing discontinuous technology industries (such as the IT industry, which has an impact on innovation in many other industries), its competitiveness seems to be much less. The above-mentioned existing institutional practices hinder the development of the market for the products and services of venture capital-backed startups.
What management decisions could create better opportunities for startups, especially software startups? By relying on spin-off companies and supporting existing partners, Japanese management has shown strong risk aversion. They are unwilling to take greater risks by partnering with startups, promising more radical innovations in order to reap greater rewards. In fact, there is a long list of Japanese tech startups that have found it necessary to go abroad in order to achieve commercial success. There are multiple lessons for policymakers in other countries. Government policymakers can work to educate corporate management and give procurement managers more leeway and incentives to buy from startups. The government can serve as a role model and change procurement policies to give startups a better chance to compete. Independent software companies have long called on the Japanese government to do this. The United States has such a policy, and New Zealand has recently followed suit, so it's a realistic option. By taking these steps, policymakers can stimulate more software startups and create a more dynamic software industry. In summary, when policymakers consider how to promote entrepreneurship, they should not only focus on stimulating supply, but also consider how business and government policies can stimulate startup growth by reducing demand-side barriers.
4. As the case of India shows, even a software industry with a strong foundation in custom-developed software can leverage the English skills of its technical community to grow its overseas software business. Japan's leaders could have put Japan on a more positive IT trajectory by enacting strict English language skills requirements for IT engineers. This will significantly expand the pool of capable Japanese IT experts, allowing them to easily access the latest global IT developments, thereby increasing the number of IT professionals who understand their importance. As discussed earlier, this will also open up export markets by making Japanese IT services more globally localized.
Perhaps more importantly, the stronger English skills of Japanese IT professionals will make acquiring foreign software companies and personnel a more viable option. Such acquisitions could have been used to inject new software knowledge and capabilities into enterprise applications and embedded software. Applied to embedded software, such acquisitions could loosen the dominance of hardware engineers and even challenge reliance on custom-developed software.
Surprisingly, in recent years, Samsung executives have started from much the same point as their Japanese counterparts, backing software acquisitions and hiring English-speaking expatriates at home and abroad to challenge their existing hardware culture and help lead their efforts to improve their software capabilities. In South Korea, both parents and the government strongly support the development of English language skills, which undoubtedly enhances their willingness to do so and the likelihood that some of these efforts will be successful. Japan generally ranks among the Asian countries with the lowest English proficiency scores. The example of South Korea reminds us that policymakers in other countries can choose to develop strong English skills, not just for those who aspire to professional careers in IT, but for the general population. The potential benefits of allowing society's young people to grow up with English language skills (and have regular access to the English-language Internet from an early age) in developing the IT industry cannot be underestimated.
In summary, analyzing what is wrong with the Japanese software industry can provide policymakers seeking to build a strong IT industry with a variety of alternative policy directions. What is discussed here is just the beginning of this effort.
Translation: Chen Guo + NetEase Youdao
Machine-translated from the Chinese original 日本软件行业衰败究竟是做错了什么？中国该如何吸取教训 (published 2024-01-13). WeChat original ↗