{"id":488779,"date":"2023-04-24T07:58:53","date_gmt":"2023-04-24T05:58:53","guid":{"rendered":"https:\/\/mybroadband.co.za\/news\/?p=488779"},"modified":"2023-04-24T14:19:04","modified_gmt":"2023-04-24T12:19:04","slug":"real-world-study-shows-generative-ai-tools-made-workers-14-more-productive","status":"publish","type":"post","link":"https:\/\/mybroadband.co.za\/news\/software\/488779-real-world-study-shows-generative-ai-tools-made-workers-14-more-productive.html","title":{"rendered":"Real-world study shows generative AI tools made workers 14% more productive"},"content":{"rendered":"

Customer service workers at a Fortune 500 software firm who were given access to generative artificial intelligence tools\u00a0became 14% more productive<\/a><\/strong>\u00a0on average than those who were not, with the least-skilled workers reaping the most benefit.<\/p>\n

That\u2019s according to a new study by researchers at Stanford University and the Massachusetts Institute of Technology who tested the impact of generative AI tools on productivity at the company over the course of a year.<\/p>\n

The research marks the first time the impact of generative AI tools on work has been measured outside the lab.<\/p>\n

Prior studies have benchmarked the capabilities of large language models against\u00a0tasks in fields like law and medicine<\/a><\/strong>\u00a0\u2014 showing that, for example,\u00a0GPT-4 aces the bar exam<\/a>\u00a0<\/strong>in the 90th percentile.<\/p>\n

Other research has tested the tech\u2019s impact on workers\u2019 performance of isolated writing tasks in\u00a0small-scale laboratory settings<\/a><\/strong>.<\/p>\n

The results of some of these earlier experiments showed the potential, at times\u00a0surprising,\u00a0of large-language models in the workplace, said Erik Brynjolfsson, the director of the Digital Economy Lab\u00a0at the Stanford Institute for Human-Centered AI.<\/p>\n

But\u00a0until the tools are tested in the real world, he said, their impact remains mostly speculative.<\/p>\n

\u201cHaving people use it for over a year in this company, you get a much better sense of how that translates into real-world productivity,\u201d Brynjolfsson,\u00a0one of the study\u2019s co-authors, said in an interview. \u201cAs far as I know, this is the first time it’s been done in a real-world setting.\u201d<\/p>\n

Brynjolfsson, alongside MIT researchers Danielle Li and Lindsey Raymond, tracked the performance of more than 5,000 customer support agents, based primarily in the Philippines, across key metrics like how quickly and successfully workers were\u00a0able to solve clients\u2019 problems.<\/p>\n

The agents were divided into groups: Some were given access to the AI tools \u2014\u00a0trained on a large set of successful customer service conversations \u2014\u00a0and others were not.<\/p>\n

The name of the company, which specializes in\u00a0enterprise software for small and medium-sized US businesses,\u00a0was not disclosed\u00a0in the report.<\/p>\n

Gains for Low-Skilled Workers<\/strong><\/h2>\n

One of the study\u2019s findings was that novice workers benefitted most from the tech, the researchers said.<\/p>\n

With the assistance of AI, the firm\u2019s least-skilled workers were able to get their work done 35% faster.<\/p>\n

New workers\u2019 performance also improved much more rapidly with the assistance of AI than without: According to the study, agents with two months of experience who were aided by AI performed just as well or better in many ways than agents with over six months of experience who worked without AI.<\/p>\n

The research suggests that the boost in low-skilled workers\u2019 productivity and performance\u00a0may come, in part, from the way that AI tools can absorb the tacit knowledge that makes the firm\u2019s top performers excel \u2014\u00a0like knowing the best language to use to soothe an irate customer\u00a0or what technical documentation would be most helpful to share in each situation \u2014 and then disseminate that knowledge to less-skilled or experienced workers through AI-generated suggested responses.<\/p>\n

These findings run counter to the prevailing notion that automation tends to hurt low-skilled workers most, as has played out over the last several decades of technological advances in manufacturing and other industries.<\/p>\n

The productivity gains \u2014\u00a0about 14% on average \u2014\u00a0were less dramatic than in prior experiments, likely because real-world workplace processes are much more complex than one-off tasks.<\/p>\n

Still, the boost in productivity was significant. \u201cThis suggests that those laboratory studies were pointing in the right direction, and that they weren’t just mirages,\u201d Brynjolfsson said.<\/p>\n

Compensation Questions<\/strong><\/h2>\n

The most highly skilled workers saw little to no benefit from the introduction of AI into their work.<\/p>\n

These top performers were likely already giving the responses at the same caliber that the AI was recommending, so there was less room for improvement \u2014\u00a0if anything, the prompts may have been a distraction, the researchers said.<\/p>\n

If AI does ultimately narrow\u00a0the gap between low- and high-skilled workers, however, companies may need to fundamentally rethink the logic underpinning\u00a0compensation choices.<\/p>\n

Top customer service agents had\u00a0Excel spreadsheets where they collected phrases that they used often and that worked well, MIT\u2019s\u00a0Raymond said.<\/p>\n

If the AI tool is indeed taking this tacit knowledge and distributing it to others, she said, \u201cthen these high-skilled workers are doing the additional service for the firm by providing these examples for the AI,\u00a0but they’re not being compensated for it.\u201d<\/p>\n

In fact,\u00a0they may be worse off\u00a0because their incentives were based on performance relative to their peers, which introduces a host of\u00a0weighty policy questions about how workers should be compensated for the value of their data.<\/p>\n

Forward-thinking companies would be wise to recognize the expertise of their star employees\u00a0since\u00a0their tacit knowledge and skill will likely\u00a0form the basis of the AI tools that will power the rest of the organization, said Brynjolfsson.<\/p>\n

\u201cSuccessful companies will have incentive and reward systems that recognize that these top performers \u2014\u00a0whether or not their performance with any given customer is demonstrably better than the less-skilled workers \u2014\u00a0create knowledge that the whole organization depends on,\u201d he said.<\/p>\n

\u201cIt wouldn’t be far-fetched for them to put even more of a premium on those people because now that kind of skill gets amplified and multiplied throughout the organization. Now that top worker could change the whole organization.\u201d<\/p>\n

Of course, Brynjolfsson noted that it\u2019s still early days in the study of generative AI and this research isn\u2019t the final word \u2014\u00a0much more is left to learn.<\/p>\n

Remaking the Workplace<\/strong><\/h2>\n

Some of the observations yielded by the field experiment weren\u2019t captured by the data but point\u00a0to dozens of other ways these tools may soon\u00a0reshape\u00a0workplaces.<\/p>\n

Anecdotally, managers at the firm were no longer spending 20 to 30 hours per week coaching employees, Raymond said, likely because the AI served as a substitute of sorts.<\/p>\n

That could, in turn, change the employee-manager relationship, since supervisors would then spend less time with direct reports and would instead take on larger teams.<\/p>\n

Still, the speed at which generative AI appears to be capable of transfiguring workplaces \u2014\u00a0seemingly overnight \u2014\u00a0is dizzying, especially relative to prior technological breakthroughs.<\/p>\n

\u201cThere\u2019s a mountain of research that these transformative technologies \u2014\u00a0like electricity or the steam engine or computers \u2014\u00a0took decades before they really moved the dial on productivity.<\/p>\n

In the case of electricity, it was about 30 years between when it was introduced into factories and when you really saw significant productivity gains,\u201d Brynjolfsson said.<\/p>\n

\u201cSo there’s a concern, even an expectation, that this would play out over the period of many years or a decade or more. But the fact that we’re already seeing it so quickly says something about the technology, and our ability to implement it and get practical results a lot faster than we did in the past.\u201d<\/p>\n

In light of these early results, Brynjolfsson has\u00a0advice for\u00a0workers and executives: Embrace this technology.<\/p>\n

\u201cStart experimenting with it and learn what it can do. Find out where it’s most effective and where it’s least effective,\u201d he said.<\/p>\n

\u201cCompanies should have crash programs to educate their workforce on them and really get up to speed.\u201d<\/p>\n

Now read: Snapchat rolls out AI chatbot to all users for free<\/a><\/h2>\n","protected":false},"excerpt":{"rendered":"

The research marks the first time the impact of generative AI tools on work has been measured outside the lab.<\/p>\n","protected":false},"author":341034,"featured_media":488791,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_sma_x_autopost_status":"idle","_sma_x_autopost_error":"","_sma_x_post_id":"","_sma_facebook_post_id":"","_sma_instagram_post_id":"","_sma_threads_post_id":"","_sma_x_attempts":0,"footnotes":""},"categories":[16],"tags":[35793,84939,30872,30298],"class_list":["post-488779","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-software","tag-artificial-intelligence-ai","tag-generative-ai","tag-massachusetts-institute-of-technology-mit","tag-stanford-university"],"_links":{"self":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts\/488779"}],"collection":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/users\/341034"}],"replies":[{"embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/comments?post=488779"}],"version-history":[{"count":1,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts\/488779\/revisions"}],"predecessor-version":[{"id":488793,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts\/488779\/revisions\/488793"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/media\/488791"}],"wp:attachment":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/media?parent=488779"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/categories?post=488779"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/tags?post=488779"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}