a package for llm self-reflection
Project description
llmReflect
llmReflect is a python package designed for large language model (LLM) applications. We have seen numerous emergent abilities so far. Given by a right prompt, a LLM is capable of various tasks. Also the art of writing a prompt usually determines the performance of the LLM at that task. So is there a chance that we can use LLM to evaluate / improve itself's prompt?
Warning! This project is at the very early stage!
Installation
-
- llmReflect is on PYPI.
pip install llmreflect
- llmReflect is on PYPI.
-
- use pipenv and git clone
git clone https://github.com/Recherches-Neuro-Hippocampe/llmReflect.git
pipenv shell
pipenv install
- use pipenv and git clone
Basic usage
1. Case 1:
Create a chain for a following workflow:
- asking questions given by a dataset
- generate postgresql cmd to solve the question
- score itself's performance
from llmreflect.Chains.DatabaseChain import DatabaseQnAGradingChain
def test_grading_chain():
uri = "your database connection uri"
ch = DatabaseQnAGradingChain.from_config(
uri=uri,
include_tables=[
'table name 1',
'table name 2',
],
open_ai_key=config('OPENAI_API_KEY')
)
logs = ch.perform(n_question=1)
2. Case 2:
How to write your own prompt and save it to your prompt base in llmReflect
from llmreflect.Prompt.BasicPrompt import BasicPrompt # import the prompt class
prompt_dict = {
"HARD": "most important rules, like general context, role acting etc.",
'SOFT': "minor rules/guide for the llm, what need to pay attention to.",
'INCONTEXT': [
# show llm some examples
{
'request': "show me the names and phone numbers of the patients that are likely to have an error in their birth date",
'command': '''\
select "uuid_patient","patient_code", "patient_first_name","patient_last_name",array_agg("phones") filter (where phones <> '{{}}') as "phones"
from tb_patient
where "patient_birth_date" > current_date
or "patient_birth_date" < '1900-01-01'
group by "uuid_patient", "patient_code","patient_first_name","patient_last_name"
limit 500;
''',
'summary': "Error: Found 0 record! Empty response!",
'grading': "9.2",
'explanation': "It is very good. It filled the blank of what could be an error in birth date by some common sense. The syntax is correct, the query is executable. Even though it got an empty response, it is because the dataset does not contain such content."
}
],
'FORMAT': { # dictionary indicating the format for llm
'request': {'type': 'INPUT',
'explanation': "User's natural language request"},
'command': {
'type': 'INPUT',
'explanation': "the generated postgresql command"
},
'summary': {
'type': 'INPUT',
'explanation': "a summary for the excution result from database"
},
'grading': {
'type': 'OUTPUT',
'explanation': "the score you assign to the postgresql command, a number range from 0 to 10"
},
'explanation': {
'type': 'OUTPUT',
'explanation': "the reason for your scoring"
},
}
}
p = BasicPrompt(
prompt_dict=prompt_dict,
promptname="gradingpostgresql" # the name is used for saving and referencing
)
p.save_prompt()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
llmreflect-0.1.8.tar.gz
(28.6 kB
view hashes)
Built Distribution
llmreflect-0.1.8-py3-none-any.whl
(37.6 kB
view hashes)
Close
Hashes for llmreflect-0.1.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 159912b8d9acb2e649bcc8c3f00be2cf5fb1e030cbbd78f7244013623b0cf7ab |
|
MD5 | efd0006e41ba06158384895278d2f899 |
|
BLAKE2b-256 | a7b7c811fec630f3407c7fd3a96aae93029ae006ec13f2fe21ef2eab590e859a |