Gpt j chatbot

The intent of this template is to serve as a quick intro guide for fellow developers looking to build AI Powered Personalized Characters

Jeanie is a GPT-J powered virtual girlfriend based on this repository

• Windows / Mac / Linux with Git installed
• Python 3.5+
• MessengerX.io API Token - FREE for Indie Developers
• Ngrok for Tunneling
• Desktop / Laptop with a minimum of 16GB RAM
• GPU is required for faster inference
• EleutherAI/gpt-neo-1.3B (Be patient, first load and / or first time inference can take time)
• NLPCloud.io Token - (Optional)

git clone git@github.com:machaao/gpt-j-chatbot.git

cd gpt-j-chatbot

pip install -r requirements.txt

nano -w .env

API_TOKEN = <Machaao API Token>
BASE_URL = <Machaao Base Url> ex: https://ganglia.machaao.com
NAME = <Your Bot Display Name> ex: Jess
DASHBOT_KEY = <Your dashbot token> (Optional) (Conversational Analytics)
MODEL_NAME=EleutherAI/gpt-neo-1.3B # for local inference / testing [we recommend using GPT-NEO 1.3B for 16GB ram - load can take up to 1 min and inference avg is about 10 - 20 seconds] See https://huggingface.co/EleutherAI for additional models
NLP_CLOUD_TOKEN = <NLP Cloud Token>  # for faster remote inference


# BOT PARAMS - Unset Parameters would use their default values.
# Don't use Top_p and Temperature parameters simultaneously.
# Default Values are mentioned below.
# The higher this value, the less deterministic the result will be
TOP_P="1.0" 
# The higher this value, the less deterministic the result will be
TEMPERATURE="0.8"
# The lower this value, the less likely GPT-J is going to generate off-topic text
TOP_K="50"
# The maximum number of tokens that the generated text should contain
MAX_LENGTH="50"

For better understanding of the above GPT-J parameters, check out the nlpcloud.io docs

• Get your FREE Developer API Token via MessengerX.io, replace it in the .env file under the entry API_TOKEN
• MessengerX.io allows you to build and integrate a custom chatbot in your website or app

• You can acquire an NLPCloud API Key via NLP Cloud and replace it in the .env file under the entry NLP_CLOUD_TOKEN

• You can acquire the API Key via Dashbot.io and replace it in the .env file under the entry DASHBOT_KEY

bot_name is a very understanding girl
bot_name and stranger are seeing each other
Here is a recent discussion between stranger and bot_name
###
stranger: hi
bot_name: hello there

def core(self, req: str, user_id: str):

• Refer to platform documentation for personalization options

python app.py

ngrok http 5000

Update your bot Webhook URL at MessengerX.io Portal with the url provided as shown below to continue development

If you use [Ngrok.io](https://ngrok.io), your webhook URL would be of the format as in the example below
https://1234-115-187-40-104.ngrok.io/machaao/hook

Visit: https://messengerx.io/<bot-name>

We are assuming you have access to a heroku account and have installed heroku command line client for your OS.

heroku login

heroku create

git commit -m ".env updated"
git push heroku master

heroku logs --tail

Update your bot Webhook URL at MessengerX.io Portal with the heroku app url

Webhook Url: <YOUR-HEROKU-APP-URL>/machaao/hook

Visit: https://messengerx.io/<your-character-name>

Tips for MPS on Mac [Use Nightly Build]
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cpu

• Running on MPS is unstable
• Running on CPU is reliable but slower
• Approx inference time > 60 seconds on Mac M1 Pro 2021 / 16GB (GPT-Neo)

• Please note that this document isn't meant to be used as a guide for production environment setup.
• Reach out to us on Twitter for any queries