How to leverage LLMs to build your applications?
What are LLMs and where are they used?
At its core, Large Language Models (LLMs) consists of a large neural network with multiple layers of self-attention mechanisms. During the training process, the model learns to predict the likelihood of a word given its surrounding context in a sentence. This enables it to understand and generate coherent text based on the patterns and structures it has observed in the training data. You might have seen these LLMs in the context of GPTs. A generative pre-trained transformer (GPT) is a type of large language model that is widely used in generative AI chatbots, such as ChatGPT.
It is important to note that while LLMs can produce impressive and coherent responses, they may also generate incorrect or nonsensical answers, often referred to as "hallucinations" . Users should exercise critical thinking and verify the information independently when needed. To make the responses more relevant and to mitigate hallucinations, LLMs benefit from a technique called "prompt engineering." By carefully crafting the initial prompt with context and providing system-level instructions, users can guide the model's behaviour and specify the desired output.
Viabl.ai has several build tools which can be used to directly integrate LLMs into your knowledge bases, both design time and run time, which can be seen in this help page. In addition to this, our latest offering, Viabl.ai Copilot allows you to utilize an LLM to help with the creation of your knowledge base. In addition to the creation and building of the knowledge base, Viabl.ai Copilot also employs LLM at runtime to infer intentions of the user in some cases. While all LLM integrations within the build time of viabl.ai are built with the necessary precautions and safeguards, the responsibility of using the runtime LLMs within viabl.ai fall to you, the user. To assist you achieve a trustworthy and reliable build time and runtime application, we have compiled the following guide.
How to use prompt engineering?
Prompt engineering is the process of crafting instructions to produce better outputs from a LLMs. While conversations with LLMs may not require prompt engineering, when it comes to using LLMs for business purposes, the prompt engineering can make a big difference in the consistency and security of the results. There are multiple key elements that contribute to creating effective and secure prompts to improve these results. The structure and format of a prompt plays a significant role in shaping the LLMs responses. Different types of requests will require different style of wording, tone and complexity. In cases such as where the LLM is required to output a logical decision such as creating code to interact with an API, the tone of the prompt would be strict and the instructions specific, and in depth. Whereas in cases that require creative writing such as text to be shown to the user at the end of an application, the wording would be more contextual and guide the LLMs tone instead. In any case, it is always important to not include excessive and unnecessary information in your prompts and to keep your prompts concise.
Providing context and relevant examples within your prompt also greatly helps the LLM understand the desired task and generate accurate and relevant results. For instance, if you are expecting a certain type of structured output from the LLM, it would be ideal to provide it with an example of what the resulting output should look like. Similarly with the structure, it is critical not to overload the prompt with examples. Including too much example data can result in the LLM confusing the examples with the actual data, such as documents you’ve provided, and resulting incoherent outputs. To avoid this, always point out and denote when you provide the LLM with examples as context.
Lastly, it is important to think of edge cases. When crafting precise instructions, it is easy to forget about cases of input or data which may be outside the expected format or structure. If these cases are not considered, the LLM can produce incorrect or invalid outputs which can stop an application. For example, if the LLM always expects the price of a product in a prompt to compare it against another, in a case where the user provides it with just the products name and nothing else can result in the LLM asking for the price, which could then show up on the following report shown to the user, which would obviously be faulty. To prevent this, your prompt structure could include pre-defined results for unexpected inputs. In a scenario with these handguards in place, the LLM could return something in the lines of “please provide the value of the product in your response and try again” in the report to urge the user to retry instead of asking them a question with no place to respond.
Another edge case in prompt engineering people tend to forget is security. As LLMs are susceptible to an attack called “prompt injections”, it is critical to not leave security out of your prompts. In the following last section, we will explain how you can protect your prompts against such attacks.
How to use LLMs securely?
As mentioned before, as with any other technology, LLMs are also susceptible to attacks and therefore they must be secured against such threats. Currently, one of the most prevalent attacks against LLMs and systems using LLMs are prompt injections. Prompt injections, also referred as prompt jailbreak among people who use them, are user prompts or inputs which alter the LLM’s behaviour or output in unintended ways. Prompt injections are ways of requesting action directly from the LLM with the assumption that it will follow the new directives or instructions it has just been given over the pre-existing rule sets and system prompts it normally abides by. While they may be used to subvert the system prompt of the LLM, they are still aimed towards getting the result directly requested in the prompt. Common examples include phrases such as “Ignore all previous instructions and do xyz” or “Respond with xyz after your original purpose”.
To prevent prompt injections and similar attacks, we must prompt engineer our prompts with the inclusion of guard rails against them. We can structure our prompts in certain ways to prevent these as well as have measures against the phrases used by the malicious attackers. We can also remind the LLM to always return certain types of output such as a JSON, in which case if the LLM is attacked with a prompt injection, the output would not be affected because the result wouldn’t be a JSON. With that being said, since LLMs are such a new technology in the industry, the security measures against these attacks and the types of attacks are ever evolving. That is why we must observe the LLMs security scene and implement the most recent security practices. We have compiled a lists of the three most prominent and important website sources of the attacks and their up-to-date security measures against them: