LLM plugin for models hosted on Replicate
Project description
llm-replicate
LLM plugin for models hosted on Replicate
Installation
Install this plugin in the same environment as LLM.
llm install llm-replicate
Configuration
You will need an API key from Replicate. You can obtain one here.
You can set that as an environment variable called REPLICATE_API_TOKEN
, or add it to the llm
set of saved keys using:
llm keys set replicate
Enter key: <paste key here>
To fetch and save details of the default collection of language models hosted on Replicate, run this:
llm replicate fetch-models
Usage
Run llm models list
to see the list of models:
llm models list
Then run a prompt through a specific model like this:
llm -m replicate-llama-7b "Ten great names for a pet pelican"
Registering extra models
To register additional models that are not included in the default Language models collection, find their ID on Replicate and use the llm replicate add
command.
For example, to add the joehoover/falcon-40b-instruct model, run this:
llm replicate add joehoover/falcon-40b-instruct --alias falcon
This adds the model with the alias falcon
- you can have 0 or more aliases for a model.
Now you can run it like this:
llm -m replicate-joehoover-falcon-40b-instruct "Three reasons to get a pet falcon"
Or using the alias like this:
llm -m falcon "Three reasons to get a pet falcon"
You can edit the list of models you have registered using the default $EDITOR
like this:
llm replicate edit-models
Development
To set up this plugin locally, first checkout the code. Then create a new virtual environment:
cd llm-palm
python3 -m venv venv
source venv/bin/activate
Now install the dependencies and test dependencies:
pip install -e '.[test]'
To run the tests:
pytest
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llm_replicate-0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cfbc8ad2e009129ef7c268c326b1bd11f3c7141ddc79c602387aac0b45008cc2 |
|
MD5 | fb5a00a489166cb387f79615bc8ebc4e |
|
BLAKE2b-256 | d76566d4098fcc0f17d7b5221d74d1c9d7c0d88afd96d584d04411fc5fc2e636 |