Chatbot Design Elements in the Era of LLMs

The journey of chatbot design has been led by advancements in AI and large language models such as GPT-4. Today, AI-driven chatbots can deliver more organic, compelling, and productive user interactions. Read our guide that describes the nuances of crafting AI-powered chatbots. Learn about new pitfalls in chatbot design and how to amp up chatbot performance.

Chike Agbai
September 27, 2023
illustration for outsourcing

Chatbot design has become top of mind again.  In this era of digital transformation driven by Generative AI and machine learning, chatbots are once again being considered potential game-changers for customer service, marketing, and internal operations.  

With the remarkable progress in artificial intelligence, particularly the introduction of large language models (LLMs) like GPT-3.5, chatbot design has experienced a significant leap forward.

By understanding the different design elements of chatbots including the methods used to design them and best practices for building chatbots, you can ensure that your AI chatbot meets user needs and aligns with your business goals.

Updating Our Views on Chatbot Design

Our journey with AI chatbot development began in 2016 when we built our very first chatbot. From our experience, successful chatbot design has hinged on the application's ability to mimic human conversation in real time and respond within context while moving the conversation or experience forward - akin to text messaging or voice interactions.

The best chatbot experiences are able to produce high quality responses that match the context of the human user. 

Just spend a few minutes with OpenAI's chatbots and you quickly understand how important they can be to a business. However, not all chatbots have as much financial backing or third-party data to back their performance in the way GPT-3.5 and its siblings do.

But today, you can easily find several online customer support chatbot examples that offer product suggestions, book reservations, place food orders, and more. Good chatbots such as HealthyScreen, tackle businesses' daily challenges effectively and quickly.

So with the introduction of modern chatbots, we thought it be good to update some tips for creating enjoyable user experiences when creating a good chatbot personality and best practices for chatbot design that will serve any developer or company seeking to introduce a modern chatbot.

In 2017, we wrote that to ensure a seamless and effective conversational experience, a well-designed chatbot should:

  1. Define its purpose
  2. Understand its deployment platform
  3. Choose the best type based on its purpose
  4. Use the right design elements
  5. Track data and analyze user behavior

Frame the Chatbot's Purpose

A chatbot’s design should first identify what potential value a given customer will gain from the chatbot. As chatbot technology has evolved, there are great use case examples where live chat and chatbots integrate into an excellent model handling customer support and communicating with customers efficiently in real-time.

But the very first thing a good chatbot should do is explain itself to the user.  In that instance, the user has a good idea of what the bot is designed to do. As a developer you can always equip the chatbot with additional powers on the backend to improve conversation performance and support capabilities. 

Understand your Chatbot’s Environment

It's critical to have a solid concept of where your chatbot will be deployed before you start creating one. The platform you choose to deploy your bot will significantly impact chatbot design, user flows, and the target audience or voice you need to engage with them. For example:

Facebook Messenger and WhatsApp

Facebook Messenger is a messaging app that lets you communicate with friends and family. Messenger can send text messages, photos, videos, and audio clips. You can also use Messenger to make voice and video calls. Messenger also has a robust chatbot ecosystem with many quick keys and tools to rapidly build a Facebook Messenger Chatbot or chatbot for WhatsApp. The Messenger apps can give your bot some superpowers that you may want to take advantage of. We highlight them in a moment.

Google Assistant

Google Assistant is the virtual assistant developed by Google that debuted on the Pixel and the Google Home smart speaker. Users can ask questions, control smart home devices, listen to news, search apps or play music. However, at the time of this update to the article, it is unclear how Google will introduce some of its many LLMs models to Google Assistant or create a reasonable platform for developers to build new applications atop Google Assistant using modern LLMs.

Amazon Alexa

Who doesn’t deep down hate the Amazon Alexa sitting in their kitchen? It stalls out on playing a new song. It butts into private conversations with unhelpful suggestions. And is seemingly only useful as "smart" timer for cooking.  Alexa has tens of thousands of "skills" that act as chatbot experiences but are powered by clearly dated technologies. We believe similar to Google it is only a matter of time before Amazon introduces a modern solution. But until then, we see no good reason to build new Alexa skills.

The Web

There's no question that the web is the platform of choice when it comes to chatbots. As such, many companies are building their own AI chatbots and integrating them into their websites.  The web remains the easiest and cleanest platform for building chatbots atop and gives you the most degrees of freedom for designing your chatbot.

Back in the Day, You Had to Choose the Best Chatbot for Your Purpose

In 2016, after you had figured out a use case for the chatbot and which messaging platform to use, you needed to consider which chatbot experience you wanted to create for your target audience. Back then the choice was between Rule-Based Chatbots and Gen 1.0 Natural Language Bots.

When we advised customers about building bots we told them up-front that a chatbot can be designed with two conversational interfaces in mind:

  • Rules-Based Chatbots, or
  • Natural Language based Chatbots.

Rule-Based Chatbots

Traditionally, chatbot design was largely a process of scripting a detailed decision tree. Each customer query was expected to follow a specific path, resulting in the bot giving a pre-scripted response. This rule-based approach often fell short, leading to a frustrating user experience when the bot encountered queries outside of its programming.

Completely scripted, rule-based bots can be built by kids using Google Sheets or professionally using the hundreds of chatbot platforms in the marketplace. There are so many to choose from that we have stopped trying to catalog them. We published a brief blog post on several of them way back in 2017, which you can find on our blog

For complete candor, we did not like to create scripted chatbots.

Why? Because rule-based chatbot tools force chatbot design into a corner from the outset. They break easily and their experiences bored the user. Unfortunately, they were easy to assemble (at least rudimentary ones), and many developers pushed these fragile snowflakes onto customers with disastrous results (practically killing the chatbot market!).

To make them work well, however, an excellent scripted chatbot required the designer and development team to have spent a lot of time thinking through the chatbot's overall personality and the type of questions they anticipated users would have.

The rules-based chatbot design process looked like a decision tree where each action by the user prompts the chatbot's responses. The approach created a spaghetti-like approach to chatbot building.

Depending on the use case, this approach led to perhaps 10-20 lines of scripted text up to hundreds of lines of scripting. In one scripted experience in 2017, we wrote over 500 lines to handle just a small set of use cases where natural language processing (NLP) would not be a good substitute.

Many chatbot developers who created scripted experiences saw their scripts grow to thousands of lines making them basically unmanageable. 

‍Natural Language Based Bots Version 1.0.

The goal when designing chatbots is to create a fluid chat experience for the end user regardless of the technical choices the development team.

Without question today the objective is to build your chatbot using artificial intelligence.  100% machine learning, AI-based chatbots that take advantage of NLP offered by LLMs like Chat GPT, variations of LLaMA and many others create unique experiences that can entertain and delight users. 

However, before these newer models, we were stuck with emerging tools from large vendors. Solutions like Lex, Cortana, and others were closed models. We could make some changes but we could never make needed changes to the core of the models to fit domain specific use cases.  Open source solutions like RASA showed promise but they still proved inadequate for building robust chatbots capable of handling more complex problems.

The new models have supplanted many of the older rigid chatbot making tools from Amazon, Google, and Facebook, making bot building easier to a degree. We strongly prefer going down this path with most bots of any complexity.

Shifting to AI-Powered Chatbot Design

‍The advent of LLMs like GPT-4 has revolutionized the chatbot design landscape. These advanced models leverage AI to understand context and generate human-like responses. This shift has significant implications for chatbot design. 

Designers can now focus more on refining the conversational abilities of the chatbot, training it on specific domains, and ensuring it provides value to the end-user, resulting in a more engaging and effective user experience.

LLM-Aided Chatbot Design Principles

Designing a chatbot is a blend of art and science, incorporating user interface design, UX principles, and AI model training. The chatbot must be designed to provide value to its users and align with the platform on which it will operate, the audience it will serve, and the tasks it will perform.

Earlier we said that a well designed chatbot should have several traits, but let's add 2 more:

  1. Define its purpose
  2. Understand its deployment platform
  3. Choose the best type based on its purpose
  4. -> Use More Machine Learning Techniques
  5. -> Make Decisions
  6. Use the right design elements
  7. Track data and analyze user behavior

More Machine Learning: Introduce LangChain into Your Design

This is more of an architectural point for developers but as you introduce different LLMs you may get different results. For instance, LangChain helps overcome many limitations of LLMs such as hallucination and limited input lengths.

Hallucination refers to where the LLM generates a response that is not supported by the input or context – meaning it will output text that is irrelevant, inconsistent, or misleading. We have had good success merging LangChain with other development techniques to get easy going chatbots that produce strong answers.

You can monitor performance through continuous conversational log review and strong maintenance of a Master Record of Truth within the chatbot. These efforts will yield a "smarter" chatbot that can operate more autonomously in the wild. 

Give the Chatbot Agency to Make Decisions

Understanding why bots fail remains crucial. Analytical insights not only enhance user experience but also shed light on potential pitfalls in chatbot design. By studying where in the user journey or conversation flow the bot falls short, we can refine and improve the design accordingly.

The use of engines or APIs for analyzing chatbot data can reveal how users interact with the bot and manage their responses. Such insights can help identify gaps in the chatbot's understanding, in its ability to guide the conversation effectively, or in the relevance of its responses.

One powerful feature is the ability to receive user feedback directly through the chatbot. For instance, the chatbot could ask users to rate their experience or offer a simple reply button for users to provide immediate feedback. This real-time feedback can inform enhancements to the bot’s design and function.

Fallback Scenarios and User Feedback

Chatbot designers need to consider various factors, including fallback scenarios that enhance the customer experience without human intervention. For instance, if a query isn’t understood by the bot, it should offer options to contact a human operator or redirect to a related FAQ section.

Direct user feedback is a powerful feature. For example, a chatbot can display a simple replies button, giving users an immediate method to provide feedback. This data is essential to refine chatbot design and make iterative improvements based on user preferences and requirements.

‍Use the Right Chatbot Design Elements

Your choice of chatbot design elements should align with the chosen deployment platform. Many chatbots employ graphic elements like cards, buttons, or quick replies to aid conversation flow. However, it's essential to ensure these graphical elements display correctly across platforms.

For instance, an SMS/text bot wouldn't support cards or buttons, whereas a bot designed for Facebook or a web interface can fully utilize these elements. Other common elements include the 'Get Started' button, Carousel, Quick Answers, Smart Reply, and Persistent Menu. These elements, used wisely, can create a smooth, user-friendly chat experience.

‍More Chatbot Design Elements

In defining the aim of chatbots, designers should consider design considerations and design options to build a practical conversational experience.

Here’s a quick snapshot of elements you can use for your bot:

  • Buttons – Buttons trigger actions when a user touches or clicks them. This is similar to any mobile app experience. By adding an interactive button, you can simplify complex workflows and help the user make decisions faster.
  • Get Started Button – For many people first experiencing a bot on messenger, this is the first thing they will see and is quite intuitive. This button revs up the bot to get going and represents a clear call-to-action for the user. This is a must-have button for Facebook Messenger Bots, for example.
  • Cards – Card layouts put images, text, buttons, links, etc. — into a series of blocked containers. These blocks can be layered or moved and tend to adjust to the size of the screen, stacking and falling into columns if you turn your phone on its side.
  • Carousel - In the design process, you can place Cards in a series where they create a carousel. A user can cycle through the cards and select the individual one that is right for them. We suggest 2-3 cards per carousel to maintain simplicity.
  • Quick Answers – Or quick replies provide a way to present bot response buttons for the user. Quick Replies appear prominently above the composer, with the keyboard-less prominent. The users' response messages are sent back to the bot when answers are tapped. Quick answers can improve the conversational experience in cases where the users have easy to address questions that can be found in a knowledge base or faq.
  • Smart Reply – Similar to those familiar with Google Inbox, smart reply gives users a fast and easy way to respond to the bot's questions without typing. Smart Reply saves time and pushes the user more efficiently through a conversation workflow. This can be particularly useful when the bot is context-aware or has some level of information at the ready about the user.
  • Persistent Menu – The Persistent Menu allows the user to navigate to another part of the bot that may not be readily available. In short, it is a way to transition into another part of the bot without having to ask the bot. A designer may use the Persistent Menu to personalize the bot or serve as a place where the user can learn more about the bot's features.

These are just a selection of popular elements that can be embedded into a bot experience. And while you can employ many or all of these on some platforms, it’s best to try to pick the option that is right for the moment.

The goal when designing chatbots is to create a fluid chat experience for the end user and customers. If not, you could run into a very cluttered and confusing experience for the user. After all the bots’ purpose is to make the user’s life simpler.

In defining the aim of chatbots, designers should consider design considerations and design options to build a practical conversational experience.

User Engagement Increases

Many bots use graphic elements like cards, buttons, or quick replies to the design flow. A visual design element helps users access key features of the bot more quickly and help users move through conversation faster.  

Messages that include pictures or graphical elements, such as contact cards and buttons, may not be displayed correctly on certain messaging platforms.

For example, if you wanted to build a bot for SMS/texting, you won’t have access to cards or buttons. But if you were creating a chatbot experience for Facebook or a web interface, you can take advantage of these options and more.

‍Tracking User Data and Behavior

As chatbot designers, we acknowledge the importance of tracking user data and analyzing behavior to enhance user experiences. A well-designed chatbot should collect data in the background to fuel iterative improvements. Data insights enable us to tailor the chatbot's tone, responses, and interaction style to best fit user preferences and requirements.

Chatbot design is a rapidly evolving field with the advent of Large Language Models like GPT-4. This new generation of AI-powered chatbots is not just functional tools, but conversational partners that drive user engagement and satisfaction to new heights. Following best practices in chatbot design, leveraging the power of LLMs, and remaining responsive to user feedback will help create more robust, intuitive, and intelligent chatbot interfaces.

Design Intelligent AI Chatbots

Chatbot design is a dynamic and evolving field that demands a keen understanding of user interactions and expectations. A well-designed chatbot leverages versatile design elements within the application but also needs to incorporate machine learning models that are able to understand context and respond seamlessly.

The emergence of Large Language Models opens a range of new design and development choices that you should consider before building your chatbot.  Today you can transform your chatbot from a mere functional tool into a conversational partner that elevates user engagement and satisfaction. By adhering to best practices in chatbot design, harnessing the power of LLMs, and remaining responsive to user feedback, designers can create more robust, intuitive, and intelligent chatbot interfaces.


We are Azumo
and we get it

We understand the struggle of finding the right software development team to build your service or solution.

Since our founding in 2016 we have heard countless horror stories of the vanishing developer, the never-ending late night conference calls with the offshore dev team, and the mounting frustration of dealing with buggy code, missed deadlines and poor communication. We built Azumo to solve those problems and offer you more. We deliver well trained, senior developers, excited to work, communicate and build software together that will advance your business.

Want to see how we can deliver for you?

schedule my call

Benefits You Can Expect

Release software features faster and maintain apps with Azumo. Our developers are not freelancers and we are not a marketplace. We take pride in our work and seat dedicated Azumo engineers with you who take ownership of the project and create valuable solutions for you.

Industry Experts

Businesses across industries trust Azumo. Our expertise spans industries from healthcare, finance, retail, e-commerce, media, education, manufacturing and more.

Illustration of globe for technology nearshore software development outsourcing

Real-Time Collaboration

Enjoy seamless collaboration with our time zone-aligned developers. Collaborate, brainstorm, and share feedback easily during your working hours.

vCTO Solution Illustration

Boost Velocity

Increase your development speed. Scale your team up or down as you need with confidence, so you can meet deadlines and market demand without compromise.

Illustration of bullseye for technology nearshore software development outsourcing

Agile Approach

We adhere to strict project management principles that guarantee outstanding software development results.

Quality Code

Benefits from our commitment to quality. Our developers receive continuous training, so they can deliver top-notch code.

Flexible Models

Our engagement models allow you to tailor our services to your budget, so you get the most value for your investment.

Client Testimonials


Azumo has been great to work with. Their team has impressed us with their professionalism and capacity. We have a mature and sophisticated tech stack, and they were able to jump in and rapidly make valuable contributions.

Drew Heidgerken
Director of Engineering

We worked with Azumo to help us staff up our custom software platform redevelopment efforts and they delivered everything we needed.

James Wilson
Discovery Channel

The work was highly complicated and required a lot of planning, engineering, and customization. Their development knowledge is impressive.

Discovery Channel
Costa Constantinou
Senior Product Manager

Azumo helped my team with the rapid development of a standalone app at Twitter and were incredibly thorough and detail oriented, resulting in a very solid product.

Seth Harris
Senior Program Manager

So much of a successful Cloud development project is the listening. The Azumo team listens. They clearly understood the request and quickly provided solid answers.

Matt Sutton
Head of Product
Bento for Business

Azumo came in with a dedicated team that quickly grasped our problem and designed and built our data integration solution. They delivered a clearer picture for our business in a timeframe I didn’t think was possible.

Bento for Business
Sean Anderson
Chief Operating Officer

How it Works

schedule my call

Step 1: Schedule your call

Find a time convenient for you to discuss your needs and goals

Step 2: We review the details

We estimate the effort, design the team, and propose a solution for you to collaborate.

Step 3: Design, Build, Launch, Maintain

Seamlessly partner with us to confidently build software nearshore

We Deliver Every Sprint

Icon illustrating the advantage of time zone-aligned software developers from Azumo, ensuring work hours synchronized with client schedules.

Time Zome Aligned

Our nearshore developers collaborate with you throughout your working day.

Icon showcasing the advantage of hiring expert engineers from Azumo for software development services.

Experienced Engineers

We hire mid-career software development professionals and invest in them.

Icon symbolizing how Azumo's software developers prioritize honest, English-always communication for building quality software.

Transparent Communication

Good software is built on top of honest, english-always communication.

Icon representing how Azumo's developers enhance velocity by approaching software development with a problem solver's mindset.

Build Like Owners

We boost velocity by taking a problem solvers approach to software development.

Icon illustrating how Azumo's quality assurance process ensures the delivery of reliable, working code for every project.

Expect Consistent Results

Our internal quality assurance process ensures we push good working code.

Icon depicting how Azumo follows strict project management principles to stay aligned with your goals throughout the development process.

Agile Project Management

We follow strict project management principles so we remain aligned to your goals