This is Part 3 of the series "Generative AI for Business Leaders"
Data has long been seen as a key competitive advantage in artificial intelligence, with more and higher quality data enabling companies to train more powerful AI models. However, recent developments in AI are changing this dynamic. Pre-trained models like Anthropic Claude and GPT-3 have already consumed massive amounts of publicly available data, reducing the advantage of proprietary data sets. New techniques also allow AI models to generate synthetic data for training. This levels the playing field for companies without access to unique data sources.
While data is becoming more commoditized, the AI model itself offers opportunities for differentiation. Foundational models like GPT provide general capabilities, performing reasonably well across many tasks like a jack-of-all-trades. However, companies can specialize these models by fine-tuning them on proprietary data related to a specific task or domain. This adjustment of parameters allows the model to become an expert for your particular needs while retaining its general knowledge. Healthcare, customer service, and other applications can all benefit from tailored fine-tuning.
But maximizing an AI system requires more than just data and models. Prompt engineering has emerged as a critical interface between humans and AI, providing the context and guidance for models like GPT-3 to generate useful, targeted outputs. Carefully crafted prompts with specific instructions and examples steer the model towards desired results. Prompt engineering also enables few-shot learning, where models can perform new tasks with minimal new data by leveraging a well-designed prompt.
In addition, as models like GPT-3 are further trained, they accumulate knowledge and skills with broad applicability, like common sense reasoning and conversational ability. This expands their capabilities on downstream tasks without needing specialized data or training for every new application. For instance, a model fine-tuned on customer support data could plausibly perform well answering queries about a new product, aided by its generalized conversational skills.
However, there are still limitations to relying solely on pre-trained models. Customer-specific data remains important for fine-tuning to maximize performance on specialized tasks. Models may also exhibit biases or make factual errors without additional training on targeted data. Maintaining proprietary local versions of large models can also be computationally expensive. Thus, while pre-trained models reduce data dependence, they do not eliminate the value of company-specific data sets.
In summary, while data remains valuable for fine-tuning, proprietary data sets are no longer an insurmountable advantage. Businesses must focus more on prompt engineering and specialized modeling to deliver unique value from AI. Those who can master the model and interface will gain a competitive edge over those who simply amass data. The future success of AI-powered companies relies on human-machine collaboration through prompts, not just data alone.
Sources:
Anthropic. (2022). Ptuning. https://www.anthropic.com
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big?. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. https://dl.acm.org/doi/abs/10.1145/3442188.3445922
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Bohg, J. (2022). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. https://arxiv.org/abs/2108.07258
Cheng, A. (2022). AI doesn't always need huge amounts of data to be useful. Harvard Business Review. https://hbr.org/2022/07/ai-doesnt-always-need-huge-amounts-of-data-to-be-useful
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586. https://arxiv.org/abs/2107.13586
Comments