Getting Started

QvikChat is a framework built on top of Firebase Genkit and LangChain. It allows you to quickly create and deploy a chat service, with support for advance features such as safety guardrails, authentication, response caching, chat history, Retrieval Augmented Generation (RAG) and more.

If you wish to get started quickly with minimal setup or if you are planning to build a standalone server to serve chat endpoints, its recommended that you start with the QvikChat Starter Template. It comes pre-configured with support for TypeScript, ESLint, Prettier, Jest, SWC, and more.

Installing QvikChat Package

You can install QvikChat using the following command:

npm install @oconva/qvikchat

pnpm add @oconva/qvikchat

Setup Environment Variables

By default, QvikChat uses Google's Gemini API for text generation and embedding models. If you don't yet have a Google Gen AI API key, you can get one from Gemini API - Get an API Key (opens in a new tab).

You can also use OpenAI API instead of Gemini API. You'll have to provide your OpenAI API key as the OPENAI_API_KEY environment variable and configure your chat endpoints to use the LLM model you want it to use.

QvikChat being built on top of Firebase Genkit, allows you to easily add Genkit plugins. You can easily use any LLM model available through any Genkit plugin by simply configuring that plugin when setting up Genkit.

To learn more about configuring chat endpoints to use a specific LLM model, check here.

If using the default Gemini API models or OpenAI models, there should be a .env file at the root level of your project directory, and it should have a value for at least one of the following, depending on which API you want to use:

GOOGLE_GENAI_API_KEY=
OPENAI_API_KEY=

Alternatively, you can copy the .env.tmp file or rename it to .env and fill in the values.

Usage

To use QvikChat, the typical workflow would look something like this:

Setup Genkit: Before you can deploy your chat endpoints, you need to setup Firebase Genkit, either by using the default configurations or by providing your configurations, these may include additional Genkit plugins you may want to enable (e.g. to add support for a new language model). When starting, we recommend using the default configurations.
Define Chat Endpoints: You can define your chat endpoints using the defineChatEndpoint method. This method allows you to easily enable or disable features like chat history, cache, authentication, and RAG. All you have to do is provide the appropriate configurations as the argument.
Run the Server: Once you have Genkit setup and the chat endpoints defined, all that's left to be done is to start the server. You can use the runServer method to start an Expressjs server. You can optionally configure the port number, cors, and other options for the server by providing the configurations in an object.

The below code shows a simple example of defining an open-ended chat endpoint in QvikChat. An open-ended chat endpoint basically allows queries related to any topic, i.e., it doesn't place any restrictions on what context the queries should be related to.

index.ts

import { runServer, setupGenkit } from "@oconva/qvikchat/genkit";
import { defineChatEndpoint } from "@oconva/qvikchat/endpoints";
 
// Setup Genkit
setupGenkit();
 
// Open-ended chat
defineChatEndpoint({
  endpoint: "chat",
});
 
// Run server
runServer();

Running the above code will run a Express (opens in a new tab) server with your defined chat endpoint accessible through it.

You should be able to access the chat endpoint defined above at the chat endpoint. To test from terminal, you can try the below command:

curl -X POST "http://127.0.0.1:3400/chat" -H "Content-Type: application/json"  -d '{"data": { "query": "Answer in one sentence: What is Firebase Firestore?" } }'

Above example points to http://127.0.0.1:3400. You can change this port and host depending on where you are running the server and on which port.

QvikChat Starter Template

To get up and running quickly, you can use the QvikChat starter template. The starter template is a pre-configured project with all the necessary configurations and setup to get you started with QvikChat write quality and reliable code. It comes pre-configured with support for TypeScript, ESLint, Prettier, Jest, SWC, and more, so you can get started with developing the next revolutionary chat app right away, without worrying about all these things.

Setup

Simply, clone the QvikChat starter template (opens in a new tab) to get started.

git clone https://github.com/oconva/qvikchat-starter-template.git

Environment Variables

Once you have cloned the starter template, add the API keys required to access LLM models. By default, QvikChat uses Google's Gemini API for text generation and embedding models. If you don't yet have a Google Gen AI API key, you can get one from Gemini API - Get an API Key (opens in a new tab).

To learn more about configuring chat endpoints to use a specific LLM model, check here.

GOOGLE_GENAI_API_KEY=
OPENAI_API_KEY=

Usage

To use QvikChat, the typical workflow would look something like this:

Setup Genkit: Before you can deploy your chat endpoints, you need to setup Firebase Genkit, either by using the default configurations or by providing your configurations, these may include additional Genkit plugins you may want to enable (e.g. to add support for a new language model). When starting, we recommend using the default configurations.
Define Chat Endpoints: You can define your chat endpoints using the defineChatEndpoint method. This method allows you to easily enable or disable features like chat history, cache, authentication, and RAG. All you have to do is provide the appropriate configurations as the argument.
Run the Server: Once you have Genkit setup and the chat endpoints defined, all that's left to be done is to start the server. You can use the runServer method to start an Expressjs server. You can optionally configure the port number, cors, and other options for the server by providing the configurations in an object.

Running the Project

QvikChat starter template comes pre-defined with some chat endpoints. These endpoints are defined in the src/endpoints directory. We setup Genkit and run the server in the src/index.ts file.

You can run the following commands to install the dependencies:

npm install # or pnpm install

To start the server, run:

npm run start # or pnpm start

Check the testing endpoints section to learn how to test endpoints.

To learn more about the QvikChat starter template, check the QvikChat Starter Template (opens in a new tab) repo.

Testing Endpoints

You can test the pre-defined test endpoints to see how the chat endpoints work and to confirm QvikChat setup. You can do this either using a graphical interface or by running the server locally and testing the endpoints using the terminal.

Genkit Developer UI

You can run the Genkit developer UI to test the endpoints. Testing the endpoints using a graphical interface is probably the easiest way to get started. You can know more about the Genkit Developer UI here (opens in a new tab).

Start the Genkit developer UI:

npx genkit start

OR, you can install the Genkit CLI globally:

npm i -g genkit

Then start the Genkit developer UI:

genkit start

You should be able to see your defined chat endpoints under the Flows section in the left sidebar. Simply click on the endpoint you want to test and enter the query you want to test with. Clicking the Run button will send the query to the endpoint and the response generation process will start.

Running the Server

You can also run the server locally to test the endpoints from their REST endpoints.

Before you can do this, you will need to first compile the TypeScript code.

Compile the TypeScript code. You can modify .swcrc to change the SWC configurations and package.json to adjust the build command.

npm run build

pnpm build

Then, start the server:

npm run start

pnpm start

Depending on which endpoint you wish to test, and where and on which port your server starts, you should be able to access the chat endpoints through the terminal using the curl command. The below given example sends the query to the chat endpoint:

curl -X POST "http://127.0.0.1:3400/chat" -H "Content-Type: application/json"  -d '{"data": { "query": "Answer in one sentence: What is Firebase Firestore?" } }'

How it works?

Each chat service is defined by defining a chat endpoint. The chat endpoint that you define, is the code that gets called when a new request is received at that endpoint. You can define multiple chat services, each with its own endpoint. Each chat endpoint can have various attributes like input data schema, LLM model to be used for processing queries, authentication policy, etc., and you can define these properties and the code that gets executed within an endpoint when defining a chat endpoint.

QvikChat provides you with an easy way to define these chat endpoints qvikly using the defineChatEndpoint function. This is the real superpower of QvikChat. You can define a chat endpoint with support for chat history, authentication and response caching in just a few lines of code.

To learn more about different chat endpoints that you can define, check the Chat Endpoints section.

Introduction Chat Endpoints