CodeT5+ (or CodeT5) is an advanced LLM designed for developers. It can generate source code, and explain what your code does. It has a good performance while being lightweight enough to run on a laptop for both inference and fine tuning, and can be trained with additional knowledge-data. I’ll show how to set up and use CodeT5+ on your laptop in minutes.

💡 Try the inference web demo. If you need the LLM to better understand your code or toolchain, you can fine-tune it. It is really not very difficult to set up, and doesn’t require expensive hardware. Read here how to do the training, and here how to use CodeT5 with LangChain.

CodeT5 vs CodeT5+?

CodeT5 is an advanced Transformer-based model designed for both understanding and generating code. It stands out by effectively handling code identifiers, leveraging user-written comments, and excelling in various code-related tasks, surpassing previous methods.

CodeT5, provided by Salesforce, comes pre-trained as small, base and large versions differing in the size of trained parameters. The newer versions are called CodeT5+ which I will use here.

Installation on Your Laptop

CodeT5+ delivers impressive performance while remaining capable of running smoothly on a local laptop for inference without any issues. Fine-tuning can also be conducted locally, making CodeT5 an ideal choice for developers to experiment with large language models (LLMs).

Setting Up Your Environment

I’ll show how to set it up and use CodeT5+ fine-tuned with some KDE code.

For an easy installation and demonstrating fine-tuning, this GitHub repo is well suitable and uses the new CodeT5+. It can run as a local server and comes with a simple demo html page, a great choice if you want to make a quick REST call from your app and go from there. And for reference, here is the original Salesforce GitHub repo.

The model is capable of describing code accurately, and you can fine-tune this model with your own code snippets to make it better or know your latest API changes.

To get started, create a dedicated folder for your project and set up the necessary Python packages. Here are the steps:

git clone https://github.com/tm243/CodeT5-KDE.git

Create a virtual environment in your project folder:

virtualenv .env

Activate the virtual environment:

source .env/bin/activate

Install the required Python packages from the provided “requirements.txt” file:

pip install -r requirements.txt

Once you’ve set up the environment, you’ll need to obtain and configure the model weights before running the CodeT5+ model. Here’s how you can do that:

Download the model weights from the specified URL:

wget https://www.opendocstring.com/downloads/weights/codet5/saved-pretrained-kde-cpp-multisum-2023-05-10-06.tar.gz

2. Unpack the downloaded model weights. You can do this manually or by using the provided script:

Manually:

tar -xzvf saved-pretrained-kde-cpp-multisum-2023-05-10-06.tar.gz

Or by using the script:

./download_weights.sh

After completing these steps, you’ll have the model weights available in the “api/saved-pretrained-kde-…” directory, and your environment will be set up to run the CodeT5 model effectively.

Using CodeT5 for Inference

To test your model you can run the inference.py script in the folder.

Though, the model becomes more useful when starting it as a local server and make requests to it. You can do:

uvicorn api.rest:app --port 7999 --reload

Now, there is a demo.html in your folder which you can open and make some queries. Try to paste some code and see what it does.

Prompt example: Write a replace method for a string class which replaces the given string with a given set of characters. Try it out on this running demo.

If you found that it does a wrong description for your code, you might want to fine-tune your model. Keep in mind that the model is trained on certain code and programming languages, but maybe hasn’t seen any code similar to the one you tried.

Fine-tuning CodeT5 (Optional)

If you want to add capabilities to the model, such as another programming language, toolkit usage or other usage patterns, you can fine-tune this model.

Why fine tuning?

Fine tuning leverages on a pre-trained model, and simply adds knowledge on top. This way, you don’t need to go through the full, lengthy and expensive training process. Fine tuning is much faster and cheaper.

How to Fine-tune CodeT5

In order to do that, you will need to create a dataset which is then fed to the model training. To get some results, a dataset of about 100 entries is already sufficient. The entries should show some variety and cover different, but similar topics.

For example, if you want to improve your models capacity to understand Python Django code, you can spend some amount on Django Views, some on Django models, and so on. They are close enough to have things in common (Django) and different enough for diversification (models, views, etc).

The data is organized in simple csv files of the following format:

"code","docstring","url","license"

The training process will then read this file and feed it to the model. Read more about preparing training data in this blog post.

I will in a next blog post show how to set up such a fine-tune training on a Macbook M1.

Let me know if you have any questions about running and fine-tuning CodeT5, either in the comment section below or reach out.