HuggingFace runtime for MLServer
Project description
HuggingFace runtime for MLServer
This package provides a MLServer runtime compatible with HuggingFace Transformers.
Usage
You can install the runtime, alongside mlserver
, as:
pip install mlserver mlserver-huggingface
For further information on how to use MLServer with HuggingFace, you can check out this worked out example.
Settings
The HuggingFace runtime exposes a couple extra parameters which can be used to
customise how the runtime behaves.
These settings can be added under the parameters.extra
section of your
model-settings.json
file, e.g.
---
emphasize-lines: 5-8
---
{
"name": "qa",
"implementation": "mlserver_huggingface.HuggingFaceRuntime",
"parameters": {
"extra": {
"task": "question-answering",
"optimum_model": true
}
}
}
These settings can also be injected through environment variables prefixed with `MLSERVER_MODEL_HUGGINGFACE_`, e.g.
```bash
MLSERVER_MODEL_HUGGINGFACE_TASK="question-answering"
MLSERVER_MODEL_HUGGINGFACE_OPTIMUM_MODEL=true
```
Reference
You can find the full reference of the accepted extra settings for the HuggingFace runtime below:
.. autopydantic_settings:: mlserver_huggingface.settings.HuggingFaceSettings
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mlserver-huggingface-1.3.1.tar.gz
(15.0 kB
view hashes)
Built Distribution
Close
Hashes for mlserver-huggingface-1.3.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6a139c2c8505c3bb4f9b3f2479e833b5c874f702b77598027c3b7c5a76b8474c |
|
MD5 | 2f6e806e02cc4b0bbd253420ba551667 |
|
BLAKE2b-256 | c3aada18b7eab99c0d38715bc1f75ad5bf92744931616a98eee8bf456e20c12e |
Close
Hashes for mlserver_huggingface-1.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0437770bb3a550e1f75d3246cd2c7109f6adcfdc310df7dc330ea028932b13f2 |
|
MD5 | be089821e90c89ea320ea6a875a77570 |
|
BLAKE2b-256 | 9bdd5679a942c669d48ce136fa459e2ab11c5b06bd7d7446f0a1aaa79c1e692e |