Patchwork AI and Organizational Technology
AI Will Be Invisible and Seamless, Not the Patchwork AI We See Today
I’m using AI to translate my posts. Check out “Patchwork AI and Organizational Technology” in: العربية | Português | Bahasa Indonesia | 简体中文 | हिन्दी | Français | Español.
Over the next year, I will share what I am learning during my sabbatical on Superadditive.co. I plan to focus on understanding modern AI, particularly how it could shape innovation, strategy, and organizational performance. If that sounds interesting, I would be honored to have you along for the journey. I will continue to post on other topics that I find interesting too…
TL;DR
AI is currently integrated into workflows through manual, fragmented, cut-and-paste methods, but it will only become transformative once it seamlessly embeds into organizational processes. This transition relies not just on whether human involvement can be effectively replaced or complemented but also on the economic incentives of software providers, firms, and users that make up the workflows inside organizations.
Today’s Reality: Patchwork AI
Today, AI exists in a patchwork state. It is manually inserted through cut-and-paste into workflows and used to assist us in thinking, searching, structuring, or translating in idiosyncratic ways.
But if AI is to have a lasting impact, and I believe it will, it will not remain visible. Like past technologies that reshaped work, it will become embedded, silent, and eventually invisible.
From Zip Drives to Overleaf: A Workflow Evolution
Writing is a large part of my work. Especially writing research papers, getting them published, and hopefully having them cited by colleagues.
When I was in graduate school, my writing process felt quite cumbersome. At that time, there was very little of what we now call “cloud computing.” After working on a manuscript in Microsoft Word, I would save it to my hard drive, copy it to a Zip drive, and open it again on my home computer.
This addition (the zip drive) to my workflow was a workaround that enabled me to transfer work from the computer lab at Carnegie Mellon to my home PC. Later, I discovered another patch: emailing myself a document so I didn’t have to rely on a Zip drive. I would work on a manuscript at home, then email it back to myself to continue the next day in the lab.
Hours wasted cleaning up citations.
As many people know, academic writing also requires citing prior literature to give credit to others whose ideas you build upon. Adding citations to a paper, let alone a dissertation, was incredibly cumbersome. I remember spending hours refining the citations after multiple rounds of revisions. I had to ensure that names were spelled correctly, dates were accurate, and that the references at the end of the manuscript adhered to the correct format, whether MLA, APA, or Chicago style, and were listed in the proper order. For every paper, I spent several hours just polishing the citations.
It was mind-numbing and tedious work. I wouldn’t wish it on anyone.
If you added it up, a significant amount of my time went into sending myself files, ensuring they were correctly numbered, copying and pasting regression tables, reformatting them to look neat, checking that citations were accurate, and performing what was, frankly, mind-numbing work.
I started using LaTeX for academic writing when I began my first academic job. LaTeX alleviated much of the headache associated with citation management. All I had to do was type \cite{…} and the citations would update automatically.
But there were still hassles with collaborative projects. We had to make sure the right file was being used and that we were not making conflicting changes.
My colleague developed a system using Subversion, a version control software for Linux. This improved things a lot.
Eventually, we transitioned from Subversion to Dropbox. Dropbox automatically syncs the file on your hard drive with the one in the cloud.
My work-from-home workflow came a long way in just 7 years: from Zip Drives to Dropbox.
Workflow Integration with Cloud Computing
Fast-forward a few more years, and I shifted my academic writing to Overleaf. Overleaf integrated what had previously been a patchwork of fixes that supported a fragmented academic writing workflow. With Overleaf, I no longer have to move files back and forth manually, update citations manually, or troubleshoot LaTeX issues.
Overleaf has saved me hundreds of hours. I estimate that for each manuscript, I save dozens of hours that were previously spent on tedious and unnecessary tasks. Now, most of my energy is focused on writing.
This transformation from Patchwork Cloud Computing to Invisible Cloud Infrastructure in the academic workflow has saved millions of hours. This wasted time was previously spent on tasks that, while necessary, did not require scholarly human capital and should not have demanded so much effort.1
Think, Search, Structure, and Translate
If we consider where generative AI stands today, we are in the early days of Patchwork AI. For most people, AI usage appears to be a series of janky insertions into workflows that have not yet been designed for its capabilities.
When we have a document that needs feedback, we paste it into ChatGPT with an off-the-cuff prompt (“vibe” prompting). We then take that output and paste it into a PowerPoint deck.
We extract text from an article, request ChatGPT to summarize or provide feedback, and paste the responses into a note-taking app. We might use AI to draft emails, check grammar, refine code snippets, brainstorm slide titles, or generate some bullet points to initiate a meeting.
In all these cases, we spend a surprisingly (though currently taken-for-granted) large portion of our limited time as cutters and pasters.
Broadly speaking, we have integrated AI into our workflows so that we can better:
Think: Reason, brainstorm, or generate a first draft. (e.g., help me explain why we should not go forward with this proposal, weigh the pros and cons).
Search: Find and explain relevant information. (e.g., what happened during the industrial revolution in India).
Structure: Organize ideas, documents, or workflows. (e.g., take these meeting minutes and highlight the most common themes)
Translate: Map your text into some other forms (e.g., can you write code to take an academic article and write it as a press release; or write code to estimate a difference-in-difference estimation with all modern methods in Stata; convert Python code into base R.)
If we observed how people use ChatGPT today, it would not look very different from watching someone emailing files back and forth. However, instead of storing and transferring information, generative AI is inserting itself into the workflow by performing these cognitive tasks that once required what we thought were inherently human skills.
From Patchwork to Invisible AI
Stepping back, it's clear that patchwork AI represents our current state. However, we will eventually transition towards something more akin to invisible AI for many of our workflows.
How and for what workflows this transition occurs is an interesting question for technology scholars, but it also has practical implications for entrepreneurs and everyday AI users.
Consider the straightforward challenge we discussed earlier: transferring files from one computer to another.
At the core of this lies an economic decision. Implementing a patch (e.g., sending a file to yourself via email) incurs some cost, C.
And there are some benefits or gains, G, that come from using the new technology. Working from home gives me more flexibility. I do not have to sit in the computer lab until 11 p.m. When I bought a Zip drive, I decided that G > C.
When I switched to emailing myself the files, I reassessed the total value created, which is V(Email) = G - C. Emailing was cheaper than buying and carrying a Zip drive. I lost some consistency because versioning became a concern. However, at each stage, I evaluated the gain versus the cost.
When I adopted Subversion, I had to spend some time learning it, but the gain was substantial: V(Subversion) > V(Email)
Notably, the value I got from adoption increased with the number of times I repeat the activity. The total value grows with each repetition N.
Now, I pay for premium Overleaf, and the benefits I gain from it are substantial. The annual cost K for the subscription is worthwhile for an integrated writing and collaboration platform. That cost begins to make sense when the value of the patchwork solution and the integrated solution start to converge as the number of times I use the solution (N) increases. At some point, the integrated solution is much value creating in the long run.
When Integration Makes Sense
On the consumer side, the equation is simple. Do I use the technology often enough that the cost of purchasing the integrated solution is lower than the total cost, in terms of time or money, associated with my patchwork approach?
Let’s assume for a moment that K for the patchwork solution is $0; for the integrated solution, it is $120. However, the net benefit of G-C for the patchwork solution is $3, and $5 is for the integrated solution.
Strategic Bottlenecks and Workflow Design
For integrated solution providers, the key question is whether the cost of developing the solution can be recouped by selling it to a sufficient number of buyers. If demand increases for specific AI workflows, integrated solutions for particular workflows are likely to emerge. This is simple math and is not very different from traditional software solution providers.
Why Humans Are Still in the Loop
The real question, however, is when the integrated human in the “patchwork” will be replaced with a fully autonomous AI solution?
I believe this depends on three factors, not all of which relate to the need for humans to be involved in the production process. It will be a mix of what humans can uniquely do, and a lot of what the incentives are for firms (both entrepreneurs and “user” firms) to invest K in developing, marketing, and centrally maintaining integrated solutions.
The Risk is Not Average Gain, But Worst-Case Loss
The first aspect relates to what humans are doing during the patchwork and whether AI can replace the entire set of steps involved in the patch. For example, if humans participate in aspects pertaining to ethics, judgment, or trust during the act of transferring information from one window to another, then establishing a direct AI pipe from App A to App B without human involvement may actually decrease the benefit, G, relative to the cost. In other words, G is some function of Humans + AI, and it cannot be replicated by AI alone.
This is particularly true if we consider G, not as some guaranteed and fixed quantity, but as a stochastic variable E[G]. That is, a payoff drawn from a distribution. What happens if your draw of a particular G is highly negative? One blunder can completely ruin the outcome. Often, the strategic value of a decision is not whether the mean is high or low, but whether you can manage the tails of the outcome distribution. For instance, a VC often makes all her money on one or two startups that go gangbusters. Similarly, HR often spends most of its time on the 1 or 2 employees who are total disasters.
Managing process variance is a crucial friction that may slow the transition into integrated AI. Visibility, and thus Patchwork AI, is a feature here even if the costs are still reasonably high.
Semi-manual patchworks are how we address uncertainties that we, as humans, want to observe and manage during our workflow. We want to be in the loop so the worst-case scenario doesn’t transpire (or we can blame someone for it!). This complicates our model slightly because it expands the G minus C equation into something that includes uncertainty around G.
Human beings participate in many workflows to reduce that uncertainty, even if it means accepting a slightly higher cost.
Complementarity and Workflow Economics
The second aspect worth considering is the extent to which the patchwork fits into a system with a high degree of complementarity. By complementarity, I mean situations where different parts of a process rely on and amplify each other’s contributions, where solving one piece doesn’t just help on its own, but unlocks progress elsewhere.
For instance, if AI can solve a problem that is an essential input into a later problem, then the impact of AI will be substantially more significant. The counterfactual is true as well. If AI makes it harder to do a downstream job, it won’t exist in the workflow for too long.
Identifying strategic bottlenecks with low risk and high gain appears to be where integration is likely to yield the greatest benefits.
Multiple Firms and Incentive Issues in the Workflow
A third factor to consider is an issue that may deserve a separate post: the incentives of the firms involved in the workflow to invest in integrating what is currently “patched.”
The typical workflow for a knowledge worker consists of software provided by various players (in a typical data I use software by Microsoft, Apple, Grammarly, Substack, Stata, R, Google, and a lot more!). The key point is that workflows are endogenous systems made up of products created by economic actors.
Consequently, there is an incentive problem for any one provider to offer something that increases value across the entire value chain.
One example that comes to mind is checking grammar.
A puzzle I have been pondering is why high-quality grammar editing isn't integrated into every single application. Why must I pay $199 a year for Grammarly? And why do I pay for that, even though Google Docs has a grammar checker, and Microsoft does too?
The fact that Grammarly has identified a market niche when grammar checks should, in theory, be integrated is very interesting (it is currently valued at around 8 billion, down from 13 billion during its last raise in 2021).
It likely has to do with the incentives, or rather the lack of them, for other providers not to invest in grammar editing modules. This is either because the fixed cost, K, of building it in-house is too high (maybe), or more likely, the pay-offs are too low.
A patchwork provider may emerge if different companies make up a knowledge worker’s workflow, and none can recover the cost of building the integrated solution internally. This provider offers significant benefits, reduces costs, and generates a positive return on investment, but the new technology use remains a patchwork rather than an integrated solution.
A Future of Mixed AI: Patchwork and Invisible
AI is likely to have both invisible and patchwork components. Invisible AI will arise from workflows dominated by a single firm or where purchasing firms find it worthwhile to invest in custom integrations. These firms are motivated to develop integrated AI due to numerous users (high Q), frequent usage (large N), and strong complementarities across the system's tasks.
At the same time, opportunities for patchwork AI will still exist. These will arise in areas where the value of human involvement is (a) high, especially due to variance issues, but too complex to automate, or (b) where interoperability issues complicate integration. If no single firm has the incentive to pay the fixed cost of building a unified solution because the integration must occur across very different and non-overlapping software environments, then the patchwork opportunities will likely persist.
Other factors, including privacy concerns, legacy software, and user trust, also make it likely that patchwork solutions will continue to play a significant role in certain areas of the AI landscape. These particular issues, however, can be folded nicely into our simple economic model.
Okay, so what’s the takeaway from all this? We currently live in a world of patchwork AI, with some glimmers of integrated solutions. Where we eventually see integrated solutions will depend on the economics of the task system, the nature of the work to be done, and the incentives of both users and firms to invest in the fixed costs required to build and adopt integrated systems.
However, learning curves will set in once adoption (whether patching or integration) occurs. It may become very difficult to displace a player like Grammarly because they have accumulated more data, experience, and have built durable user habits. So, even if an integrated solution is theoretically possible, a patchwork solution may remain more appealing due to the weak incentives for other firms to catch up and the relatively high G-C for the patchwork solution resulting from learning-by-doing effects for users.
The Value of Humans in The AI-Enabled Workflow?
Stepping back, the next set of questions we should ask is: where will humans fit in, and what is in the G that makes human involvement especially valuable? In my next post, I will explore AI as a player in the labor market. A recent article in the Wall Street Journal highlights that Moderna has decided to merge its technology and HR departments. At a broader level, this suggests that some firms already see humans and technology, particularly AI, as substitutes. They are actively working to determine the optimal mix between the two to maximize performance.
Join me next week as I delve deeper into considering benchmarks for both AI and humans, and what they might signify for the future of work as companies opt for a blend of human and AI contributions in their decision-making.
What are some Patchwork AI solutions you’ve developed for your workflow?
The cloud computing example is just one case. Often, when new technologies first appear, their use shows up as patchwork. Consider the calculator. While people still use calculators, very few carry them around for everyday math. Most of the math that people need to do now happens automatically. It is integrated. I do not know anyone my age who balances a checkbook. All the calculations required to track your bank balance happen in the background. You do not have to think about them. You barely notice them.
Of course, there is a bigger story to tell here. When we give up direct control over computation, and it becomes invisible, it raises questions about who holds the power. When calculation becomes something we no longer see, we also lose visibility into how it works and who it serves.