An Introduction to AI Systems Design
In the fast-paced world of artificial intelligence (AI) and large language models (LLMs), it's an exciting time to be part of the evolution. With the development speed of AI technologies like ChatGPT, we're witnessing just the beginning of what is likely a new exponential growth curve in innovation and capabilities. Sam Altman, CEO of OpenAI, intriguingly remarked in a recent interview that the impressive GPT-4 model "kinda sucks" compared to what's coming next. This observation may seem casual, but it underscores the rapid obsolescence within AI advancements. GPT-2 and GPT-3, which were groundbreaking not long ago, received little public attention upon release, hinting at the staggering improvements expected in future iterations like GPT-5 and beyond.
At ONOW, our year-long exploration into the practicality of using ChatGPT as a one-stop solution revealed its limitations even in straightforward tasks. Most of us have seen the humorous failures of ChatGPT with simple math, or at providing technically correct, but ineffective and general business advice that sounds like a basic textbook when asking questions like "what should I do to improve my business today?"
In this article, I attempt to explain some of the higher-level considerations about the pros and cons of using ChatGPT in everyday business development work. While incredibly powerful, we have found that ChatGPT alone only scratches the surface of possibility, and combining it with traditional engineering and product development can unlock nearly limitless possibilities that we're only beginning to explore. I aim to explore the more technical and detailed aspects of implementation in a future post.
What is ChatGPT?
ChatGPT, at its core, is a versatile tool trained on a vast amount of text data, capable of generating content, answering questions, and simulating conversation with surprising human-like proficiency. However, it's not without its limitations, and is a little like a Swiss Army knife that is a jack-of-all-trades and master-of-none. It tends to provide generalized answers and struggles with simple computational tasks and multi-step problems. Moreover, the security concerns of sending sensitive information to OpenAI cannot be overlooked, as this data could be used for future model training, posing potential security risks.
Addressing ChatGPT's Shortcomings
To mitigate these issues, strategies like utilizing Azure's OpenAI API have been implemented at ONOW for enhanced security and data privacy. Using these APIs protect any data from being used for future model training and potential security breaches. Another approach involves hosting open-source models like Mistral or Llama, though this adds complexity and cost.
Beyond security, the adoption of tools like LangChain and an agent architecture mark a significant advancement. This strategy employs a swarm-like collection of AI systems, each specializing in simple tasks. These systems work in turns or parallel to combine a fabric of complex reasoning and pieces within a broader problem-solving framework. It's a little bit like a supercharged loop statement in traditional coding that will execute many times until a pre-specified objective is achieved.
This AI orchestration can be augmented with tools like machine translation (MT), search engines, vector databases, internal databases, code generation, and tested prompt engineering, tailoring outputs to specific languages and user needs. This massively increases the potential power of the system, and many researchers are exploring an iteration of this path as they research how artificial general intelligence (AGI) may emerge.
In such systems, the GPT-based coordination is like a central controller—much like a "traffic cop"—that deconstructs a complex question into simpler, manageable parts. These parts are then assigned to various specialized agents within a collective network. Some tasks may involve secure interactions with internal databases to access company-specific information, which remains beyond ChatGPT's reach. After processing, the outcomes from these agents are compiled, merged, and refined into a cohesive response to the original, complex question. Subsequently, the system evaluates this consolidated answer, generating and appraising multiple variations to identify the most accurate response. This intricate process necessitates numerous independent interactions with many tools, of which GPT is one of many. Future articles will delve deeper into the nuances of this agent-based model, offering a comprehensive technical breakdown of its mechanics and advantages.
*Example of a basic web scraping and summarization agent using some pre-built tools from Langchain.
More than Text
While agents excel in backend processes and generating responses, they don't constitute a holistic solution. A key factor in ChatGPT's success lies in its incredibly simple and adaptable frontend interface. This user-friendly chat interface makes it a versatile tool for a wide array of applications, though it's primarily fine-tuned for basic interactions. Its design allows for easy navigation through previous dialogues and the initiation of new ones, but it isn't designed to optimize for much beyond that. Furthermore, certain aspects of its operation, such as the degree of engagement ("chattiness") and its design for responding to user inquiries rather than initiating the flow of the conversation are fixed attributes that are established behind-the-scenes.
Our early version of the AI financial tracker takes the same GPT technology used by ChatGPT but adds in more tools and a new look to make tracking money simpler and more suited to what business owners need. It's a lot like ChatGPT because it uses a chat interface, but it's specially made to keep track of money changes better than the basic ChatGPT does. This tool starts conversations about money coming in and going out of a business, aiming to figure out where at least 90% of those changes come from. It's different from ChatGPT because it focuses tightly on money matters, making it more helpful for this specific job. Once it finishes finding out where the money changes come from, it organizes this financial info, makes it easy to understand, and shows it in a way that business owners can quickly use to make decisions, all without needing them to do any extra steps.
We created this tool with the goal of making financial tracking much easier for business owners. We want it to be simple to use, cutting down on the usual hassles of keeping track of finances and removing the need for ledger-based tools. It's designed to give immediate, useful advice based on the specific financial information entered by the user, not just general tips, and a simple conversation is immeidately turned into useful graphs, charts, and transaction logs for the business owner to review and learn from.
While GPT is the important linchpin to the tool, the way we're using it here is quite different from the usual ChatGPT interactions. But this is just the start, and we believe there's a lot more we can do with AI technology when we think creatively about how to design and build these systems for business development around the world.
Beyond ChatGPT: A Platform Ecosystem Approach
Understanding ChatGPT as just one tool among many in a broader platform ecosystem shows the importance of customization and development in AI applications. The future of AI in business lies not in single, generalized models like ChatGPT, but in the strategic integration of specialized systems designed to address specific needs. Agents, data security, traditional application design and user experience are all needed to reach our goals. Many articles talk about generalized "AI" like it will supercede traditional engineering. But at this stage of development, it is clearly a powerful tool that can be combined and integrated with existing product development along others. We see incredible potential for this integration in revolutionizing business development and support in developing countries around the world.