Using Github API (to pull individuals, orgs, and article content), setup a local sqlite database, syncing images to Cloudflare.
Project description
corpus-pax
Setting up the initial, foundational tables with generic users, organizations, and articles.
flowchart LR
subgraph main
local(local machine)--run corpus-pax--->db[(sqlite.db)]
local--avatar.jpeg---cf(cloudflare)
end
subgraph github/corpus
folder1(members)--data via api---local
folder2(orgs)--data via api---local
end
subgraph github/lawsql-articles
folder3(github/lawsql-articles/content)--data via api---local
end
Implies updated Github repositories:
corpus
(for entities, i.e. members and orgs) andlawsql-articles
(markdown style articles).
With respect to entities, data concerning members will be pulled from such repository. Each avatar image should be named avatar.jpeg
so that these can be uploaded to Cloudflare.
Install
poetry add corpus-pax
poetry update
Supply .env
Create an .env file to create/populate the database. See sample .env highlighting the following variables:
- Cloudflare
CF_ACCT
- Cloudflare
CF_TOKEN
- Github
GH_TOKEN
DB_FILE
(sqlite)
Note the workflow where the secrets are included for Github actions. Ensure these are set in the repository's <url-to-repo>/settings/secrets/actions
, making the proper replacements when the tokens for Cloudflare and Github expire.
Notes
Why Github
The names and profiles of individuals and organizations are stored in Github. These are pulled into the application via an API call requiring the use of a personal access token.
Why Cloudflare Images
Individuals and organizations have images stored in Github. To persist and optimize images for the web, I use Cloudflare images.
Why sqlite
The initial data is simple. This database however will be the foundation for a more complicated schema. Sqlite seems a better fit for experimentation and future embeddability of the same for app use.
Steps
Review database connection
Use the database path declared in DB_FILE
to establish an sqlite3 connection:
from sqlpyd import Connection # this is sqlite-utils' Database() under the hood
c = Connection() # type: ignore
# the filename will later be created in the root directory of the project folder
Connection(DatabasePath="ex.db", WALMode=False)
This connection will be used to access the database.
Add persons
Create and populate the persons-related tables:
from corpus_pax import init_persons
# This is a shortcut to: init_person_tables(c); add_individuals_from_api(c); and add_organization_from_api(c)
init_persons(c)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for corpus_pax-0.1.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a9e730c9e3779ee5a1c111536ff26ff501408094281166ce449d11cf6a7ca77e |
|
MD5 | 0b381c6230a8f7f177aa03b3ab67ef5c |
|
BLAKE2b-256 | 48250a9cc7f835307e226ae77713446907cdd1d0cd420ef31d2a056e5a0d8440 |