Rebuilding SmarterChild on Twitter with Python, GPT3 and Replit

If you’re a 90s baby then you remember messing with SmarterChild on your desktop computer, that probably looked something like this.

Smarterchild was ahead of its time. Now, with the emergence of OpenAI and GPT3, we’re ready to rebuild our old friend, with a new engine under the hood.

Pre-requisites:

OpenAI API Key
Twitter Developer Account
Twitter Bot Username
Tweepy
Python
Replit

You can find my code for this post on Replit.

Authenticate With OpenAI

The first step is to create a free OpenAI account. While the account creation is free, in order to make calls to the GPT3 engine, you will need to add your billing info.

I recommend using the Davinci model for your Twitter Chatbot. It will run you about $0.0200 / 1K tokens. Said another way, two cents per ~750 words. You can see more pricing details here.

After you sign up and add your billing info, you’re ready to access your OpenAI API Key.

Copy the secret key and store it within your new Python Repl. If you’re asking yourself, “what the heck is a Repl?” then you need to spend some time diving deeper into the world of Replit.

Replit handles environment variables in a nice and easy manner. Simply create a new secret, call it OPENAI_KEY and paste your copied OpenAI API secret key into the value.

You can also store it in an .env file if you’re running this locally. You’ll need to use python-dotenv if you go this route.

Within the Repl main.py file, you’ll need to access the secret key as an environment variable. You can do so with the following code:

my_secret = os.environ[‘OPENAI_KEY’]

You need to replace my_secret with openai.api_key so it looks like this:

openai.api_key = os.environ[‘OPENAI_KEY’]

Here you are setting up the openai API client by by retrieving the environment variable named OPENAI_KEY and setting it as the openai.api_key. This API key is used to authenticate and authorize the client making the API requests to GPT3.

Create a Twitter Developer Account

Head over to the Twitter Developer Portal, assuming it still exists as you’re readying this, and create a new developer account for free (or until Elon makes you pony up some dough).

I am writing this while Elon is in the process of removing the free tier but YOLO.

Go through the steps to create a new app in the Developer Environment. After you’ve given your app a name, description and app icon, head over to the Keys and Tokens tab.

There are 4 authentication tokens you need to copy:

Consumer Key
Consumer Secret
Access Token
Access Token Secret

Just like with the OpenAI API Key, copy and paste your Twitter authentication tokens as Replit secrets. Give them the following names:

You then need to retrieve the secrets in your main.py python file:

consumer_key = os.environ[‘TWITTER_CONSUMER_KEY’]
consumer_secret = os.environ[‘TWITTER_CONSUMER_SECRET’]
access_token = os.environ[‘TWITTER_ACCESS_TOKEN’]
access_token_secret = os.environ[‘TWITTER_ACCESS_TOKEN_SECRET’]

Use Tweepy to Interact with Twitter API

For this tutorial you’ll use Tweepy as a wrapper for the Twitter API. You can find Tweepy’s documentation here.

Authenticate With Tweepy

After you import your authentication keys, tokens and secrets you need to initialize OAuthUserHandler with all four credentials and then create an api object with the OAuthUserHandler instance.

# Authenticate using the keys
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

# Create an API class object
api = tweepy.API(auth)

The newly created api class object gives you access to all of the Twitter API methods. These methods accept various parameters and return different responses. For your reply bot purpose, you’ll use the mentions_timeline() method.

Get User Mentions

Now you’re ready to access all of the tweets that mention a specific username. The bot username will be associated with your Twitter authentication tokens so you don’t need to reference it in the python file.

As mentioned earlier, you’ll call the mentions_timeline() method on the api class object to get all the mentions of your bots username. Save it into a statuses variable so we can loop over it in a for loop. You may find it helpful to print the number of statuses retrieved with a handy-dandy print statement.

# fetch the mentions with mentions_timeline
statuses = api.mentions_timeline()
print(str(len(statuses)) + “ bot mentions”)

Since this is a new bot, there likely wont be many mentions. But fear not, just fire up your burner accounts, pour yourself some red wine and start tweeting away. For more info on how to use the mentions_timeline() method check out Tweepys documentation here.

The next step is to loop over all the tweets within the statuses variable and extract the status.id for each. With the id you can use the get_status() method to get the full text of each tweet. This is necessary for Tweets with longer than 180 characters or other fancy embeddings. Save the extended_status.full_text , which is really just the entiretweet text, into a prompt variable.

for status in statuses:  
	# fetching the status with extended tweet_mode  
    id = status.id  
    extended_status = api.get_status(id, tweet_mode=”extended”)  
    prompt = extended_status.full_text  
    print(prompt)

Awesome, you now have the id’s and the text for each tweet mentioning your bot. You’ll use this as the prompt input for GPT3.

Generate Twitter Replies with GPT3

To bring your bot to life, you’ll use the Completion.create() method on the openai client you authenticated earlier. This method is used to generate a response for any given input, which in your case, is a Tweet.

The Completion.create() method requires the following parameters:

prompt
max_tokens
temperature
engine

Since your bot is tweeting, the max_tokens needs to be 280 since Twitter doesn’t accept tweets of longer length. The temperature parameter will determine how wild or stiff you want the bot to be.

Save the generated response into an r variable and clean up the response so you just have the text, which you’ll feed back into tweepy as a reply.

r = openai.Completion.create(  
	engine="text-davinci-003",  
    prompt=prompt,  
    max_tokens=280,  
    temperature=0.5, 
    )
    
# clean up response
response = r.choices[0].text.strip()

If you print the response, you should see GPT3 at work. Congrats on making it this far, you’re almost there.

Next, pass the response into Tweepy’s update_status() method which will tweet on behalf of your bot. Make sure to pass in the in_reply_to_status_id parameter so it replies to the correct Twitter mention that was used as the prompt text.

# Respond to the tweet 
api.update_status(“@” + status.user.screen_name + “ “ + response, in_reply_to_status_id=status.id)

If you run this and check Twitter, you should see your bot’s response!

Store Processed Tweets in SQLlite Database

The script should be working great at this point. But what you’ll likely see is your bot responding to the same tweets over and over again. To prevent this, you need to create a database to store the ids of the tweets your bot has already responded to.

Create a new sqlite database in replit by merely running:

# Create the database and connection
connection = sqlite3.connect(“my_database”)

Replit will automatically download sqlite3.

From there its best practice to create a cursor for the database so you can execute statements.

# Create a cursor for the database to execute statements
cursor = connection.cursor()

Create a “tweets” table which includes id, user and the tweet text. The id will be the primary key, while the user and text are nice-to-haves.

# create tweets table if it doesn’t 
existcursor.execute( 
“”” 
	CREATE TABLE IF NOT EXISTS tweets ( 
    id integer PRIMARY KEY, 
    user text, 
    text text 
    ); 
    “””
   )

After the table is created, you can run SQL statements to get all the tweet ids in the database and save it into a result variable:

# get all of the tweet ids in the database
cursor.execute(“SELECT id from tweets”)
result = cursor.fetchall()

If you were to print result you would notice that it’s not an object you can easily loop over in python.

To get around this, you should convert it to a list. The easiest way I’ve found to do that is to first turn it into a dataframe and then convert the id column of the dataframe into a list. Save this new list as processed_mentions.

# convert result to a dataframe and create a list
df = pd.DataFrame(result, columns=[‘id’])
processed_mentions = df[‘id’].to_list()

Now, before responding to each tweet, you can loop over the processed_mentions and check if the tweet you grabbed from the api.mentions_timeline() method has already been processed. If it has, you can skip it with continue :

if status.id in processed_mentions:  
	print(“skipping status “ + str(status.id))  
    continue

The last piece of the puzzle is creating a function to add the newly replied to tweets into the Tweets database. For this, create a new function called add_to_processed() which takes in status as the parameter.

With the status, set the id, user and text columns equal to the id, user.screen_name and text of the status, aka tweet:

# function to add status to sqlite3 database
def add_to_processed(status):  
	insert_query = (“INSERT INTO tweets (id,user,text)”  
    “VALUES (:id, :user, :text);”)    
    
    tweet_parameters = {   
    	'id': status.id,   
        'user': status.user.screen_name,   
        'text': status.text  
     }    
     
     return connection.execute(insert_query, tweet_parameters)

Now when you update your for loop that loops over each status within statuses, you can pass that status into the add_to_processed() function. This will insert the relevant info into the database so it’s skipped the next time around. Your final code should look something like this.

for status in statuses:  
	# skip status if its in processed mentions  
    if status.id in processed_mentions:    
    	print(“skipping status “ + str(status.id))    
        continue    

	# fetching the status with extended tweet_mode  
    id = status.id  
    extended_status = api.get_status(id, tweet_mode="extended")
    add_to_processed(status)  
    prompt = extended_status.full_text  
    
    r = openai.Completion.create(   
    	engine="text-davinci-003",   
        prompt=prompt,   
        max_tokens=280,   
        temperature=0.5,   
   		)  
        
    # clean up response  
    response = r.choices[0].text.strip()   
    
    # Respond to the tweet  
    api.update_status("@" + status.user.screen_name + " " + response, in_reply_to_status_id=status.id)

Great! The last bit of sqlite3 housekeeping is running the commit and close operations at the end of your main.py file.

# Commit changes
connection.commit()

# Close the connection
connection.close()

Voila. You should now have a working Twitter reply bot that takes in a mention, uses it as a prompt for GPT3, replies to that tweet using tweepy and then stores the id of that tweet into a sqlite3 database on replit.

If Elon hasn’t burned twitter into the ground yet, enjoy.

As always, you can find video walkthroughs for each of my side projects on my YouTube Channel, Learn With Jabe.

Cheers,

Jabe

How to Build a Twitter Chatbot With GPT3 and Replit