bert-for-tf2 0.1.4

A TensorFlow 2.0 Keras implementation of BERT.

This repo contains a TensorFlow v2 Keras implementation of google-research/bert with support for loading of the original pre-trained weights, and producing activations numerically identical to the one calculated by the original model.

The implementation is build from scratch using only basic tensorflow operations, following the code in google-research/bert/modeling.py (but skipping dead code and applying some simplifications). It also utilizes kpe/params-flow to reduce common Keras boilerplate code (related to passing model and layer configuration arguments).

LICENSE

MIT. See License File.

Install

bert-for-tf2 is on the Python Package Index (PyPI):

pip install bert-for-tf2

Usage

TBD

Resources

• BERT - BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

• google-research/bert - the original BERT implementation

• kpe/params-flow - A Keras coding style for for reducing Keras boilerplate code in custom layers by utilizing kpe/py-params