Instructions to use retrain-pipelines/function_caller_lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use retrain-pipelines/function_caller_lora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="retrain-pipelines/function_caller_lora")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("retrain-pipelines/function_caller_lora", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use retrain-pipelines/function_caller_lora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "retrain-pipelines/function_caller_lora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "retrain-pipelines/function_caller_lora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/retrain-pipelines/function_caller_lora

SGLang

How to use retrain-pipelines/function_caller_lora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "retrain-pipelines/function_caller_lora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "retrain-pipelines/function_caller_lora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "retrain-pipelines/function_caller_lora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "retrain-pipelines/function_caller_lora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio new

How to use retrain-pipelines/function_caller_lora with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for retrain-pipelines/function_caller_lora to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for retrain-pipelines/function_caller_lora to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for retrain-pipelines/function_caller_lora to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="retrain-pipelines/function_caller_lora",
    max_seq_length=2048,
)

Docker Model Runner
How to use retrain-pipelines/function_caller_lora with Docker Model Runner:
```
docker model run hf.co/retrain-pipelines/function_caller_lora
```

Aurelien-Morgan-Bot commited on Feb 27

Commit

0f30bbe

verified ·

1 Parent(s): 495f634

source-code for model version v0.37_20260227_192656740_UTC- retrain-pipelines 0.1.2

Browse files

Files changed (2) hide show

v0.37_20260227_192656740_UTC/requirements.txt +739 -0
v0.37_20260227_192656740_UTC/retraining_pipeline.py +2265 -0

v0.37_20260227_192656740_UTC/requirements.txt ADDED Viewed

	@@ -0,0 +1,739 @@

+absl-py==2.4.0
+accelerate==1.1.1
+access==1.1.10.post3
+affine==2.4.0
+aiofiles==24.1.0
+aiohappyeyeballs==2.6.1
+aiohttp==3.13.3
+aiosignal==1.4.0
+aiosqlite==0.22.1
+alabaster==1.0.0
+albucore==0.0.24
+albumentations==2.0.8
+ale-py==0.11.2
+alembic==1.18.4
+altair==5.5.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+antlr4-python3-runtime==4.9.3
+anyio==4.12.1
+anywidget==0.9.21
+apsw==3.51.2.0
+apswutils==0.1.2
+argon2-cffi==25.1.0
+argon2-cffi-bindings==25.1.0
+array_record==0.8.3
+arrow==1.4.0
+arviz==0.22.0
+astropy==7.2.0
+astropy-iers-data==0.2026.2.23.0.48.33
+asttokens==3.0.1
+astunparse==1.6.3
+atpublic==5.1
+attrs==25.4.0
+audioread==3.1.0
+Authlib==1.6.8
+autograd==1.8.0
+babel==2.18.0
+backcall==0.2.0
+beartype==0.22.9
+beautifulsoup4==4.13.5
+betterproto==2.0.0b6
+bigframes==2.35.0
+bigquery-magics==0.10.3
+bitsandbytes==0.44.1
+bleach==6.3.0
+blinker==1.9.0
+blis==1.3.3
+blobfile==3.2.0
+blosc2==4.0.0
+bokeh==3.8.2
+boto3==1.42.58
+botocore==1.42.58
+Bottleneck==1.4.2
+bqplot==0.12.45
+branca==0.8.2
+brotli==1.2.0
+CacheControl==0.14.4
+cachetools==6.2.6
+catalogue==2.0.10
+certifi==2026.1.4
+cffi==2.0.0
+chardet==5.2.0
+charset-normalizer==3.4.4
+clarabel==0.11.1
+click==8.3.1
+click-plugins==1.1.1.2
+cligj==0.7.2
+cloudpathlib==0.23.0
+cloudpickle==3.1.2
+cmake==3.31.10
+cmdstanpy==1.3.0
+colorcet==3.1.0
+colorlover==0.3.0
+colour==0.1.5
+comm==0.2.3
+community==1.0.0b1
+confection==0.1.5
+cons==0.4.7
+contourpy==1.3.3
+cramjam==2.11.0
+cryptography==43.0.3
+cucim-cu12 @ https://pypi.nvidia.com/cucim-cu12/cucim_cu12-26.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
+cuda-bindings==12.9.4
+cuda-core==0.3.2
+cuda-pathfinder==1.3.5
+cuda-python==12.9.4
+cuda-toolkit==12.8.1
+cudf-cu12==26.2.1
+cudf-polars-cu12==26.2.1
+cufflinks==0.17.3
+cuml-cu12==26.2.0
+cupy-cuda12x==14.0.1
+curl_cffi==0.14.0
+cuvs-cu12 @ https://pypi.nvidia.com/cuvs-cu12/cuvs_cu12-26.2.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl
+cvxopt==1.3.2
+cvxpy==1.6.7
+cycler==0.12.1
+cyipopt==1.5.0
+cymem==2.0.13
+Cython==3.0.12
+dask==2026.1.1
+dask-cuda==26.2.0
+dask-cudf-cu12==26.2.1
+dataproc-spark-connect==1.0.2
+datasets==4.0.0
+db-dtypes==1.5.0
+dbus-python==1.2.18
+debugpy==1.8.15
+decorator==5.2.1
+defusedxml==0.7.1
+deprecation==2.1.0
+diffusers==0.36.0
+dill==0.3.8
+distributed==2026.1.1
+distributed-ucxx-cu12==0.48.0
+distro==1.9.0
+dlib==19.24.6
+dm-tree==0.1.9
+docstring_parser==0.17.0
+docutils==0.21.2
+dopamine_rl==4.1.2
+duckdb==1.3.2
+earthengine-api==1.5.24
+easydict==1.13
+editdistance==0.8.1
+eerepr==0.1.2
+einops==0.8.2
+en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl#sha256=1932429db727d4bff3deed6b34cfc05df17794f4a52eeb26cf8928f7c1a0fb85
+entrypoints==0.4
+esda==2.8.1
+et_xmlfile==2.0.0
+etils==1.13.0
+etuples==0.3.10
+executing==2.2.1
+Farama-Notifications==0.0.4
+fastai==2.8.7
+fastapi==0.133.0
+fastcore==1.12.16
+fastdownload==0.0.7
+fastjsonschema==2.21.2
+fastlite==0.2.4
+fastprogress==1.1.5
+fasttransform==0.0.2
+ffmpy==1.0.0
+filelock==3.24.3
+fiona==1.10.1
+firebase-admin==6.9.0
+Flask==3.1.3
+flatbuffers==25.12.19
+flax==0.11.2
+folium==0.20.0
+fonttools==4.61.1
+fqdn==1.5.1
+frozendict==2.4.7
+frozenlist==1.8.0
+fsspec==2025.3.0
+future==1.0.0
+gast==0.7.0
+gcsfs==2025.3.0
+GDAL==3.8.4
+gdown==5.2.1
+geemap==0.35.3
+geocoder==1.38.1
+geographiclib==2.1
+geopandas==1.1.2
+geopy==2.4.1
+giddy==2.3.8
+gin-config==0.5.0
+gitdb==4.0.12
+GitPython==3.1.46
+glob2==0.7
+google==3.0.0
+google-adk==1.25.1
+google-ai-generativelanguage==0.6.15
+google-api-core==2.30.0
+google-api-python-client==2.190.0
+google-auth==2.47.0
+google-auth-httplib2==0.3.0
+google-auth-oauthlib==1.2.4
+google-cloud-aiplatform==1.138.0
+google-cloud-appengine-logging==1.8.0
+google-cloud-audit-log==0.4.0
+google-cloud-bigquery==3.40.1
+google-cloud-bigquery-connection==1.20.0
+google-cloud-bigquery-storage==2.36.2
+google-cloud-bigtable==2.35.0
+google-cloud-core==2.5.0
+google-cloud-dataproc==5.25.0
+google-cloud-datastore==2.23.0
+google-cloud-discoveryengine==0.13.12
+google-cloud-firestore==2.23.0
+google-cloud-functions==1.22.0
+google-cloud-iam==2.21.0
+google-cloud-language==2.19.0
+google-cloud-logging==3.13.0
+google-cloud-monitoring==2.29.1
+google-cloud-pubsub==2.35.0
+google-cloud-resource-manager==1.16.0
+google-cloud-secret-manager==2.26.0
+google-cloud-spanner==3.63.0
+google-cloud-speech==2.36.1
+google-cloud-storage==3.9.0
+google-cloud-trace==1.18.0
+google-cloud-translate==3.24.0
+google-colab @ file:///colabtools/dist/google_colab-1.0.0.tar.gz
+google-crc32c==1.8.0
+google-genai==1.64.0
+google-generativeai==0.8.6
+google-pasta==0.2.0
+google-resumable-media==2.8.0
+googleapis-common-protos==1.72.0
+googledrivedownloader==1.1.0
+gradio==5.50.0
+gradio_client==1.14.0
+grain==0.2.15
+graphviz==0.21
+greenlet==3.3.2
+groovy==0.1.2
+grpc-google-iam-v1==0.14.3
+grpc-interceptor==0.15.4
+grpcio==1.67.1
+grpcio-health-checking==1.67.1
+grpcio-status==1.71.2
+grpclib==0.4.9
+gspread==6.2.1
+gspread-dataframe==4.0.0
+gym==0.25.2
+gym-notices==0.1.0
+gymnasium==1.2.3
+h11==0.16.0
+h2==4.3.0
+h5netcdf==1.8.1
+h5py==3.15.1
+hdbscan==0.8.41
+hf-xet==1.3.0
+hf_transfer==0.1.9
+highspy==1.13.1
+holidays==0.91
+holoviews==1.22.1
+hpack==4.1.0
+html5lib==1.1
+httpcore==1.0.9
+httpimport==1.4.1
+httplib2==0.31.2
+httptools==0.7.1
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==0.27.1
+humanize==4.15.0
+hyperframe==6.1.0
+hyperopt==0.2.7
+ibis-framework==9.5.0
+idna==3.11
+ImageIO==2.37.2
+imageio-ffmpeg==0.6.0
+imagesize==1.4.1
+imbalanced-learn==0.14.1
+immutabledict==4.3.1
+importlib_metadata==8.7.1
+importlib_resources==6.5.2
+imutils==0.5.4
+inequality==1.1.2
+inflect==7.5.0
+iniconfig==2.3.0
+intel-cmplr-lib-ur==2025.3.2
+intel-openmp==2025.3.2
+ipyevents==2.0.4
+ipyfilechooser==0.6.0
+ipykernel==7.2.0
+ipyleaflet==0.20.0
+ipyparallel==8.8.0
+ipython==8.21.0
+ipython-genutils==0.2.0
+ipython-sql==0.5.0
+ipytree==0.2.2
+ipywidgets==7.7.1
+isoduration==20.11.0
+itsdangerous==2.2.0
+jaraco.classes==3.4.0
+jaraco.context==6.1.0
+jaraco.functools==4.4.0
+jax==0.7.2
+jax-cuda12-pjrt==0.7.2
+jax-cuda12-plugin==0.7.2
+jaxlib==0.7.2
+jedi==0.19.2
+jeepney==0.9.0
+jieba==0.42.1
+Jinja2==3.1.6
+jiter==0.13.0
+jmespath==1.1.0
+joblib==1.5.3
+jsonpatch==1.33
+jsonpickle==4.1.1
+jsonpointer==3.0.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+jupyter-console==6.6.3
+jupyter-events==0.12.0
+jupyter-leaflet==0.20.0
+jupyter_client==8.8.0
+jupyter_core==5.9.1
+jupyter_kernel_gateway @ git+https://github.com/googlecolab/kernel_gateway@b134e9945df25c2dcb98ade9129399be10788671
+jupyter_server==2.14.0
+jupyter_server_terminals==0.5.4
+jupyterlab_pygments==0.3.0
+jupyterlab_widgets==3.0.16
+jupytext==1.19.1
+kaggle==1.7.4.5
+kagglehub==0.3.13
+keras==3.10.0
+keras-hub==0.21.1
+keras-nlp==0.21.1
+keyring==25.7.0
+keyrings.google-artifactregistry-auth==1.1.2
+kiwisolver==1.4.9
+langchain==1.2.10
+langchain-core==1.2.15
+langgraph==1.0.9
+langgraph-checkpoint==4.0.0
+langgraph-prebuilt==1.0.8
+langgraph-sdk==0.3.9
+langsmith==0.7.6
+lark==1.3.1
+lazy_loader==0.4
+libclang==18.1.1
+libcudf-cu12==26.2.1
+libcugraph-cu12==26.2.0
+libcuml-cu12==26.2.0
+libcuvs-cu12==26.2.0
+libkvikio-cu12==26.2.0
+libpysal==4.14.1
+libraft-cu12==26.2.0
+librmm-cu12==26.2.0
+librosa==0.11.0
+libucx-cu12==1.19.0
+libucxx-cu12==0.48.0
+lightgbm==4.6.0
+linkify-it-py==2.0.3
+llvmlite==0.43.0
+locket==1.0.0
+logical-unification==0.4.7
+lxml==6.0.2
+Mako==1.3.10
+mapclassify==2.10.0
+Markdown==3.10.2
+markdown-it-py==4.0.0
+MarkupSafe==3.0.3
+matplotlib==3.10.0
+matplotlib-inline==0.2.1
+matplotlib-venn==1.1.2
+mcp==1.26.0
+mdit-py-plugins==0.5.0
+mdurl==0.1.2
+metaflow==2.19.20
+mgwr==2.2.1
+miniKanren==1.0.5
+missingno==0.5.2
+mistune==3.2.0
+mizani==0.13.5
+mkl==2025.3.1
+ml_dtypes==0.5.4
+mlxtend==0.23.4
+mmh3==5.2.0
+momepy==0.11.0
+more-itertools==10.8.0
+moviepy==1.0.3
+mpmath==1.3.0
+msgpack==1.1.2
+multidict==6.7.1
+multipledispatch==1.0.0
+multiprocess==0.70.16
+multitasking==0.0.12
+murmurhash==1.0.15
+music21==9.9.1
+namex==0.1.0
+narwhals==2.17.0
+natsort==8.4.0
+nbclassic==1.3.3
+nbclient==0.10.4
+nbconvert==7.17.0
+nbformat==5.10.4
+ndindex==1.10.1
+nest-asyncio==1.6.0
+networkx==3.6.1
+nibabel==5.3.3
+nltk==3.9.1
+notebook==6.5.7
+notebook_shim==0.2.4
+numba==0.60.0
+numba-cuda==0.22.2
+numexpr==2.14.1
+numpy==2.4.2
+nvidia-cublas-cu12==12.1.3.1
+nvidia-cuda-cccl-cu12==12.9.27
+nvidia-cuda-cupti-cu12==12.1.105
+nvidia-cuda-nvcc-cu12==12.5.82
+nvidia-cuda-nvrtc-cu12==12.1.105
+nvidia-cuda-runtime-cu12==12.1.105
+nvidia-cudnn-cu12==9.1.0.70
+nvidia-cufft-cu12==11.0.2.54
+nvidia-curand-cu12==10.3.2.106
+nvidia-cusolver-cu12==11.4.5.107
+nvidia-cusparse-cu12==12.1.0.106
+nvidia-libnvcomp-cu12==5.1.0.21
+nvidia-ml-py==13.590.48
+nvidia-nccl-cu12==2.21.5
+nvidia-nvimgcodec-cu12==0.7.0.11
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvtx-cu12==12.1.105
+nvtx==0.2.14
+nx-cugraph-cu12 @ https://pypi.nvidia.com/nx-cugraph-cu12/nx_cugraph_cu12-26.2.0-py3-none-any.whl
+oauth2client==4.1.3
+oauthlib==3.3.1
+omegaconf==2.3.0
+onemkl-license==2025.3.1
+openai==2.23.0
+opencv-contrib-python==4.13.0.92
+opencv-python==4.13.0.92
+opencv-python-headless==4.13.0.92
+openpyxl==3.1.5
+opentelemetry-api==1.38.0
+opentelemetry-exporter-gcp-logging==1.11.0a0
+opentelemetry-exporter-gcp-monitoring==1.11.0a0
+opentelemetry-exporter-gcp-trace==1.11.0
+opentelemetry-exporter-otlp-proto-common==1.38.0
+opentelemetry-exporter-otlp-proto-http==1.38.0
+opentelemetry-proto==1.38.0
+opentelemetry-resourcedetector-gcp==1.11.0a0
+opentelemetry-sdk==1.38.0
+opentelemetry-semantic-conventions==0.59b0
+opt_einsum==3.4.0
+optax==0.2.7
+optree==0.19.0
+orbax-checkpoint==0.11.33
+orjson==3.11.7
+ormsgpack==1.12.2
+osqp==1.1.1
+overrides==7.7.0
+packaging==26.0
+pandas==2.2.2
+pandas-datareader==0.10.0
+pandas-gbq==0.30.0
+pandas-stubs==2.2.2.240909
+pandocfilters==1.5.1
+panel==1.8.7
+param==2.3.2
+parso==0.8.6
+parsy==2.2
+partd==1.4.2
+patsy==1.0.2
+peewee==4.0.0
+peft==0.14.0
+pexpect==4.9.0
+pickleshare==0.7.5
+pillow==11.3.0
+pip3-autoremove==2.0.1
+platformdirs==4.9.2
+plotly==5.24.1
+plotnine==0.14.5
+pluggy==1.6.0
+plum-dispatch==2.7.1
+pointpats==2.5.5
+polars==1.35.2
+polars-runtime-32==1.35.2
+pooch==1.9.0
+portpicker==1.5.2
+preshed==3.0.12
+prettytable==3.17.0
+proglog==0.1.12
+progressbar2==4.5.0
+prometheus_client==0.24.1
+promise==2.3
+prompt_toolkit==3.0.52
+propcache==0.4.1
+prophet==1.3.0
+proto-plus==1.27.1
+protobuf==5.29.6
+psutil==5.9.5
+psycopg2==2.9.11
+psygnal==0.15.1
+ptyprocess==0.7.0
+PuLP==3.3.0
+pure_eval==0.2.3
+py-cpuinfo==9.0.0
+py4j==0.10.9.9
+pyarrow==18.1.0
+pyasn1==0.6.2
+pyasn1_modules==0.4.2
+pycairo==1.29.0
+pycocotools==2.0.11
+pycparser==3.0
+pycryptodomex==3.23.0
+pydantic==2.12.3
+pydantic-settings==2.13.1
+pydantic_core==2.41.4
+pydata-google-auth==1.9.1
+pydot==4.0.1
+pydotplus==2.0.2
+PyDrive2==1.21.3
+pydub==0.25.1
+pyerfa==2.0.1.5
+pygame==2.6.1
+pygit2==1.19.1
+Pygments==2.19.2
+PyGObject==3.48.2
+PyJWT==2.11.0
+pylibcudf-cu12==26.2.1
+pylibcugraph-cu12==26.2.0
+pylibraft-cu12==26.2.0
+pymc==5.28.0
+pynndescent==0.6.0
+pyogrio==0.12.1
+pyomo==6.10.0
+PyOpenGL==3.1.10
+pyOpenSSL==24.2.1
+pyparsing==3.3.2
+pyperclip==1.11.0
+pyproj==3.7.2
+pysal==25.7
+pyshp==3.0.3
+PySocks==1.7.1
+pyspark==4.0.2
+pytensor==2.38.0
+pytest==8.4.2
+python-apt==0.0.0
+python-box==7.4.1
+python-dateutil==2.9.0.post0
+python-dotenv==1.2.1
+python-fasthtml==0.12.47
+python-json-logger==4.0.0
+python-louvain==0.16
+python-multipart==0.0.22
+python-slugify==8.0.4
+python-snappy==0.7.3
+python-utils==3.9.1
+pytz==2025.2
+pyviz_comms==3.0.6
+PyWavelets==1.9.0
+PyYAML==6.0.3
+pyzmq==26.2.1
+quantecon==0.11.0
+raft-dask-cu12==26.2.0
+rapids-dask-dependency==26.2.0
+rapids-logger==0.2.3
+rasterio==1.5.0
+rasterstats==0.20.0
+ratelim==0.1.6
+referencing==0.37.0
+regex==2025.11.3
+requests==2.32.4
+requests-oauthlib==2.0.0
+requests-toolbelt==1.0.0
+requirements-parser==0.9.0
+# Editable install with no version control (retrain-pipelines==0.0.0)
+-e /content/pkg_src
+rfc3339-validator==0.1.4
+rfc3986-validator==0.1.1
+rfc3987-syntax==1.1.0
+rich==14.3.3
+rmm-cu12==26.2.0
+roman-numerals==4.1.0
+roman-numerals-py==4.1.0
+rpds-py==0.30.0
+rpy2==3.5.17
+rsa==4.9.1
+rtree==1.4.1
+ruff==0.15.2
+s3transfer==0.16.0
+safehttpx==0.1.7
+safetensors==0.7.0
+scikit-image==0.25.2
+scikit-learn==1.6.1
+scipy==1.16.3
+scooby==0.11.0
+scs==3.2.11
+seaborn==0.13.2
+SecretStorage==3.5.0
+segregation==2.5.3
+semantic-version==2.10.0
+Send2Trash==2.1.0
+sentence-transformers==5.2.3
+sentencepiece==0.2.1
+sentry-sdk==2.53.0
+setuptools==80.10.2
+shap==0.50.0
+shapely==2.1.2
+shellingham==1.5.4
+simple-parsing==0.1.8
+simplejson==3.20.2
+simsimd==6.5.13
+six==1.17.0
+sklearn-compat==0.1.5
+sklearn-pandas==2.2.0
+slicer==0.0.8
+smart_open==7.5.1
+smmap==5.0.2
+sniffio==1.3.1
+snowballstemmer==3.0.1
+sortedcontainers==2.4.0
+soundfile==0.13.1
+soupsieve==2.8.3
+soxr==1.0.0
+spacy==3.8.11
+spacy-legacy==3.0.12
+spacy-loggers==1.0.5
+spaghetti==1.7.6
+spanner-graph-notebook==1.1.8
+spglm==1.1.0
+Sphinx==8.2.3
+sphinxcontrib-applehelp==2.0.0
+sphinxcontrib-devhelp==2.0.0
+sphinxcontrib-htmlhelp==2.1.0
+sphinxcontrib-jsmath==1.0.1
+sphinxcontrib-qthelp==2.0.0
+sphinxcontrib-serializinghtml==2.0.0
+spint==1.0.7
+splot==1.1.7
+spopt==0.7.0
+spreg==1.8.5
+SQLAlchemy==2.0.47
+sqlalchemy-spanner==1.17.2
+sqlglot==25.20.2
+sqlite-web==0.7.1
+sqlparse==0.5.5
+srsly==2.5.2
+sse-starlette==3.2.0
+stack-data==0.6.3
+stanio==0.5.1
+starlette==0.52.1
+statsmodels==0.14.6
+stringzilla==4.6.0
+stumpy==1.13.0
+sympy==1.13.1
+tables==3.10.2
+tabulate==0.9.0
+tbb==2022.3.1
+tblib==3.2.2
+tcmlib==1.4.1
+tenacity==9.1.4
+tensorboard==2.19.0
+tensorboard-data-server==0.7.2
+tensorflow==2.19.0
+tensorflow-datasets==4.9.9
+tensorflow-hub==0.16.1
+tensorflow-metadata==1.17.3
+tensorflow-probability==0.25.0
+tensorflow-text==2.19.0
+tensorflow_decision_forests==1.12.0
+tensorstore==0.1.81
+termcolor==3.3.0
+terminado==0.18.1
+text-unidecode==1.3
+textblob==0.19.0
+tf-slim==1.1.0
+tf_keras==2.19.0
+thinc==8.3.10
+threadpoolctl==3.6.0
+tifffile==2026.2.20
+tiktoken==0.12.0
+timm==1.0.25
+tinycss2==1.4.0
+tobler==0.13.0
+tokenizers==0.20.3
+toml==0.10.2
+tomlkit==0.13.3
+toolz==0.12.1
+torch==2.5.1+cu121
+torchaudio==2.5.1+cu121
+torchcodec==0.10.0+cu128
+torchdata==0.11.0
+torchsummary==1.5.1
+torchtune==0.6.1
+torchvision==0.20.1+cu121
+tornado==6.5.1
+tqdm==4.67.3
+traitlets==5.14.3
+traittypes==0.2.3
+transformers==4.46.2
+treelite==4.6.1
+treescope==0.1.10
+triton==3.1.0
+trl==0.12.0
+tsfresh==0.21.1
+tweepy==4.16.0
+typeguard==4.5.1
+typer==0.24.1
+typer-slim==0.24.0
+types-pytz==2025.2.0.20251108
+types-setuptools==80.10.0.20260124
+typing-inspection==0.4.2
+typing_extensions==4.15.0
+tyro==1.0.8
+tzdata==2025.3
+tzlocal==5.3.1
+uc-micro-py==1.0.3
+ucxx-cu12==0.48.0
+umap-learn==0.5.11
+umf==1.0.3
+unsloth @ git+https://github.com/unslothai/unsloth.git@0c8c5ed81e423658ab9ae81eac5aab8d18f5d7af
+unsloth_zoo==2024.11.5
+uri-template==1.3.0
+uritemplate==4.2.0
+urllib3==2.5.0
+uuid_utils==0.14.1
+uvicorn==0.41.0
+uvloop==0.22.1
+vega-datasets==0.9.0
+wandb==0.25.0
+wasabi==1.1.3
+watchdog==6.0.0
+watchfiles==1.1.1
+wcwidth==0.6.0
+weasel==0.4.3
+webcolors==25.10.0
+webencodings==0.5.1
+websocket-client==1.9.0
+websockets==15.0.1
+Werkzeug==3.1.6
+wheel==0.46.3
+widgetsnbextension==3.6.10
+wordcloud==1.9.6
+wrapt==2.1.1
+wsproto==1.3.2
+wurlitzer==3.1.1
+xarray==2025.12.0
+xarray-einstats==0.10.0
+xformers==0.0.29.post1
+xgboost==3.2.0
+xlrd==2.0.2
+xxhash==3.6.0
+xyzservices==2025.11.0
+yarl==1.22.0
+ydf==0.15.0
+yellowbrick==1.5
+yfinance==0.2.66
+zict==3.0.0
+zipp==3.23.0
+zstandard==0.25.0

v0.37_20260227_192656740_UTC/retraining_pipeline.py ADDED Viewed

	@@ -0,0 +1,2265 @@

+from unsloth import FastLanguageModel, \
+    is_bfloat16_supported, UnslothTrainer, \
+    UnslothTrainingArguments
+import torch
+import os
+import gc
+import re
+import sys
+import json
+import time
+import shutil
+import logging
+import builtins
+import importlib.util
+from enum import Enum
+from textwrap import dedent
+from datetime import datetime, \
+    timezone
+import polars as pl
+from polars.exceptions import ComputeError
+from jinja2 import Environment, FileSystemLoader
+from huggingface_hub import list_repo_commits
+from datasets import load_dataset, \
+    Dataset, DatasetDict
+from datasets.config import HF_DATASETS_CACHE, \
+    HF_CACHE_HOME
+from transformers import AutoTokenizer
+from retrain_pipelines import __version__
+from retrain_pipelines.dataset.hf_utils import \
+    get_lazy_df, get_column_info, \
+    iterable_dataset_multi_buffer_sampler, \
+    push_dataset_version_to_hub
+from retrain_pipelines.dataset.tool_calls import \
+    count_tool_occurrences, plot_tools_occurences, \
+    column_words_stats, plot_words_count, \
+    get_unique_tools
+from retrain_pipelines.utils.hf_utils import \
+    get_repo_version, get_new_repo_minor_version, \
+    push_files_to_hub_repo_branch
+from retrain_pipelines.dag_engine.core import \
+    TaskPayload, task, dag, DagParam, ctx, UiCss
+from retrain_pipelines.dag_engine.rp_logging import \
+    rp_redirect_stdout
+from retrain_pipelines.dag_engine.sdk import \
+    ExecutionsIterator
+from retrain_pipelines.utils import create_requirements
+#--- helpers ----------------------------------------------------------------------------
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.DEBUG)
+class LocalServeReadinessEnum(Enum):
+    """
+    tracking local-serve (infra-validation)
+    status using a "3+"-states enum :
+        - "-1" for "not applicable"
+          (i.e. "model version not blessed"),
+        - "0/1" bool for failure/success.
+    """
+    NOT_APPLICABLE = -1
+    FAILURE = 0
+    FAILURE_NO_DOCKER = 2
+    SUCCESS = 1
+def clear_gc():
+    """Convenience method to clear
+    the content of the garbage collector.
+    Forcing it to actually clear
+    any cuda tensor it holds.
+    """
+    for obj in gc.get_objects():
+        try:
+            if torch.is_tensor(obj) and obj.is_cuda:
+                del obj
+        except:
+            pass
+    gc.collect()
+#--- retraining-pipeline elements -------------------------------------------------------
+@task
+def start() -> TaskPayload:
+    logger.info(f"{ctx.pipeline_name} - {ctx.exec_id}")
+    logging.getLogger("retrain_pipelines").setLevel(logging.INFO)
+    # inputs validation
+    repo_id_pattern = re.compile(
+        r"""
+        ^                           # start
+        (?!.*\.\.)                  # no '..' anywhere
+        (?!.*--)                    # no '--' anywhere
+        (?:                         # legacy: single segment OR namespace/repo
+            [A-Za-z0-9._-]+         # legacy: gpt2, bert-base-uncased, etc.
+            |
+            [A-Za-z0-9._-]+/[A-Za-z0-9._-]+   # namespace/repo_name
+        )
+        $                           # end
+        """,
+        re.VERBOSE
+    )
+    ctx.hf_dataset = json.loads(ctx.hf_dataset)
+    assert repo_id_pattern.match(ctx.hf_dataset["repo_id"]) is not None, \
+           f"Invalid repo_id format: {ctx.hf_dataset['repo_id']!r}"
+    ctx.augmentation_rate = float(ctx.augmentation_rate)
+    ctx.hf_enrich_dataset = json.loads(ctx.hf_enrich_dataset)
+    assert repo_id_pattern.match(ctx.hf_enrich_dataset["repo_id"]) is not None, \
+           f"Invalid repo_id format: {ctx.hf_enrich_dataset['repo_id']!r}"
+    ctx.enrichment_rate = float(ctx.enrichment_rate)
+    assert repo_id_pattern.match(ctx.dataset_repo_id) is not None, \
+           f"Invalid repo_id format: {dataset_repo_id!r}"
+    assert ctx.polars_engine in ["gpu", "cpu"]
+    ctx.hf_base_model = json.loads(ctx.hf_base_model)
+    assert repo_id_pattern.match(ctx.hf_base_model["repo_id"]) is not None, \
+           f"Invalid repo_id format: {ctx.hf_base_model['repo_id']!r}"
+    ctx.cpt_training_args = json.loads(ctx.cpt_training_args)
+    ctx.sft_training_args = json.loads(ctx.sft_training_args)
+    assert repo_id_pattern.match(ctx.model_repo_id) is not None, \
+           f"Invalid repo_id format: {model_repo_id!r}"
+    # GPU availability
+    logger.info(torch.cuda.get_device_name(0))
+    logger.info(torch.__version__)
+    ctx.engine = "cpu" if (
+                    ctx.polars_engine == "gpu" and
+                    not torch.cuda.is_available()
+                 ) else ctx.polars_engine
+    logger.debug(f"Polars engine : {ctx.engine}")
+    # hf_dataset
+    hf_dataset_dict = \
+        get_lazy_df(
+            repo_id=ctx.hf_dataset["repo_id"],
+            commit_hash=ctx.hf_dataset["commit_hash"],
+            config_name=(
+                ctx.hf_dataset["config_name"] and
+                "" < ctx.hf_dataset["config_name"]
+            ),
+            hf_token=os.getenv("HF_TOKEN", None)
+        )
+    try:
+        logger.info(f"hf_dataset_dict lazy_df : {hf_dataset_dict['lazy_df']}")
+        logger.info(
+            f"{hf_dataset_dict['repo_id']}, " +
+            f"{hf_dataset_dict['commit_hash']}  -  " +
+            f"{hf_dataset_dict['commit_datetime']}\n" +
+            hf_dataset_dict["lazy_df"].explain()
+        )
+    except ComputeError as ex:
+        if "HF_TOKEN" not in os.environ:
+            logger.info("Does the Hugging Face-hosted dataset " +
+                  "require authentication ?",
+                  file=sys.stderr, flush=True)
+        raise ex
+    hf_dataset_version = get_repo_version(
+        repo_id=hf_dataset_dict["repo_id"],
+        revision=hf_dataset_dict["commit_hash"],
+        repo_type="dataset",
+        hf_token=os.getenv("HF_TOKEN", None)
+    )
+    hf_dataset_dict["version_label"] = (
+        f"{hf_dataset_version[0]}.{hf_dataset_version[1]}"
+        if sum(hf_dataset_version) > 0
+        else None
+    )
+    ctx.hf_dataset_dict = hf_dataset_dict
+    # hf_enrich_dataset
+    hf_enrich_dataset_dict = \
+        get_lazy_df(
+            repo_id=ctx.hf_enrich_dataset["repo_id"],
+            commit_hash=ctx.hf_enrich_dataset["commit_hash"],
+            config_name=(
+                ctx.hf_enrich_dataset["config_name"] and
+                "" < ctx.hf_enrich_dataset["config_name"]
+            ),
+            hf_token=os.getenv("HF_TOKEN", None)
+        )
+    hf_enrich_dataset_version = get_repo_version(
+        repo_id=hf_enrich_dataset_dict["repo_id"],
+        revision=hf_enrich_dataset_dict["commit_hash"],
+        repo_type="dataset",
+        hf_token=os.getenv("HF_TOKEN", None)
+    )
+    hf_enrich_dataset_dict["version_label"] = (
+        f"{hf_enrich_dataset_version[0]}.{hf_enrich_dataset_version[1]}"
+        if sum(hf_enrich_dataset_version) > 0
+        else None
+    )
+    logger.info(' ; '.join(f"{k}: {hf_enrich_dataset_dict[k]}"
+                                   for k in ['commit_hash',
+                                             'commit_datetime']))
+    ctx.hf_enrich_dataset_dict = hf_enrich_dataset_dict
+    # hf_base_model
+    hf_base_model_revision=(
+        None if (rev_commit_hash:=ctx.hf_base_model["commit_hash"]) == ""
+        else rev_commit_hash
+    )
+    hf_base_model_commit = list_repo_commits(
+            repo_id=ctx.hf_base_model["repo_id"],
+            revision=hf_base_model_revision,
+            repo_type="model",
+            token=os.getenv("HF_TOKEN", None)
+        )[0]
+    # version major+minor=0 for non retrain-pipelines models
+    hf_base_model_version = get_repo_version(
+        repo_id=ctx.hf_base_model["repo_id"],
+        revision=hf_base_model_revision,
+        repo_type="model",
+        hf_token=os.getenv("HF_TOKEN", None)
+    )
+    ctx.hf_base_model_dict = {
+        "repo_id": ctx.hf_base_model["repo_id"],
+        "version_label": (
+            f"{hf_base_model_version[0]}.{hf_base_model_version[1]}"
+            if sum(hf_base_model_version) > 0
+            else None
+        ),
+        "commit_hash": hf_base_model_commit.commit_id,
+        "commit_datetime": \
+            hf_base_model_commit.created_at
+    }
+    ctx.model_version_blessed = False
+    ctx.current_blessed_exec = None
+    ctx.current_blessed_version_dict = None
+    ctx.retrain_pipelines = f"retrain-pipelines {__version__}"
+    ctx.retrain_pipeline_type = os.environ["retrain_pipeline_type"]
+    ctx.serving_artifacts_local_folder = os.path.realpath(os.path.join(
+        os.path.dirname(__file__), "..", "..", "serving_artifacts",
+        ctx.pipeline_name, str(ctx.exec_id)
+    ))
+    if not os.path.exists(ctx.serving_artifacts_local_folder):
+        os.makedirs(ctx.serving_artifacts_local_folder)
+    ctx.unsloth_dir = os.path.join(
+        ctx.serving_artifacts_local_folder,
+        "Unsloth"
+    )
+    logger.debug(f"unsloth_dir : {ctx.unsloth_dir}")
+    ctx.cpt_model_dir = os.path.join(ctx.unsloth_dir, "cpt_model")
+    ctx.sft_model_dir = os.path.join(ctx.unsloth_dir, "sft_model")
+    return None
+@task
+def eda(_) -> None:
+    """
+    exploratory data analysis.
+    """
+    ############################
+    #    features and label    #
+    #       basic counts       #
+    ############################
+    ctx.records_count = ctx.hf_dataset_dict["lazy_df"] \
+        .select(pl.len()).collect(engine=ctx.engine).item()
+    ctx.data_schema = get_column_info(
+        ctx.hf_dataset_dict["lazy_df"], engine=ctx.engine)
+    ############################
+    ############################
+    #          Answers         #
+    #        tools count       #
+    ############################
+    struct_schema = pl.Struct([
+        pl.Field("name",
+                 pl.String
+                ),
+        pl.Field("arguments",
+                 pl.List(pl.String)  # we retrieve list of args names
+                                     # (without assigned values)
+                )
+    ])
+    tool_answer_occurrences_df = \
+        count_tool_occurrences(
+            ctx.hf_dataset_dict["lazy_df"],
+            ctx.hf_dataset["attributes"]["answers_attr"],
+            struct_schema) \
+        .collect(engine=ctx.engine)
+    print(f"{tool_answer_occurrences_df['occurrences'].sum():,} " +
+          f"query/tool-calls pairs")
+    fig = plot_tools_occurences(tool_answer_occurrences_df,
+                                title_prefix="Dataset answers - ")
+    ctx.answers_tools_count_fig = fig
+    ############################
+    ############################
+    #           Query          #
+    #        words count       #
+    ############################
+    queries_max_length = ctx.hf_dataset_dict["lazy_df"].select(
+        pl.col(
+            ctx.hf_dataset["attributes"]["query_attr"]
+        ).str.len_chars().max().alias("max_query_length")
+    ).collect(engine=ctx.engine)
+    print(f"longuest query counts " +
+          f"{queries_max_length['max_query_length'][0]:,} characters")
+    # queries length quartiles
+    ctx.query_words_stats = \
+        column_words_stats(
+            ctx.hf_dataset_dict["lazy_df"],
+            ctx.hf_dataset["attributes"]["query_attr"]
+        ).collect(engine=ctx.engine)
+    print(ctx.query_words_stats.to_pandas().to_string(index=False))
+    print("Two thirds of the records have a query with less than " +
+          f"{ctx.query_words_stats['q3'][0]} words.")
+    fig = plot_words_count(
+            ctx.hf_dataset_dict["lazy_df"],
+            column_name=ctx.hf_dataset["attributes"]["query_attr"],
+            engine=ctx.engine)
+    ctx.words_count_fig = fig
+    ############################
+    ############################
+    #     hf_enrich_dataset    #
+    #    Query words count     #
+    ############################
+    enrich_question_words_stats = \
+        column_words_stats(
+            ctx.hf_enrich_dataset_dict['lazy_df'],
+            ctx.hf_enrich_dataset["query_attribute"],
+            column_attr_handler=eval(
+                ctx.hf_enrich_dataset["query_attribute_handler"])
+        ).collect(engine=ctx.engine)
+    print(enrich_question_words_stats.to_pandas()
+            .to_string(index=False))
+    del enrich_question_words_stats
+    ############################
+    return None
+@task
+def augment_data(_) -> None:
+    """
+    Add 'negative' examples, where
+    queries do not trigger any tool call.
+    To achieve that, we sample long user queries,
+    truncate at half words count, and
+    associate this to an empty list of tool-calls.
+    """
+    """
+    We only consider :
+      - records with longuest queries,
+        i.e. queries in the last quartile
+        of "queries with most word-counts"
+        (this is to avoid that 'truncated' queries
+         get really short)
+      - records with answers consisting
+        in a single tool-call
+        (in order to minimize the risk
+         that truncating actually gives
+         a valid answer with
+         one tool-call [or more])
+    Note on flow 'augmentation_rate' :
+        we add that many records (at most),
+        as quartiles size permits.
+    """
+    print("Sampling within the population with more than " +
+          str(ctx.query_words_stats['q3'][0]) +
+          " words (longest queries quartile) =>")
+    samples_count = \
+        int(ctx.records_count * ctx.augmentation_rate)
+    print(f"{ctx.augmentation_rate:.1%} would represent " +
+          f"{samples_count:,.0f} records to be sampled")
+    eligible_records_df = \
+        ctx.hf_dataset_dict["lazy_df"].filter(
+            pl.col(
+                ctx.hf_dataset["attributes"]["query_attr"]
+            )
+            .str.extract_all(r"\w+")
+            .map_elements(
+                lambda arr: len(arr),
+                return_dtype=pl.Int16)
+            .gt(ctx.query_words_stats['q3'][0])
+            & pl.col("answers")
+            .map_elements(
+                lambda x: len(json.loads(x)) == 1
+                          if isinstance(x, str)
+                          else False,
+                return_dtype=pl.Boolean)
+        ) \
+        .collect(engine=ctx.engine)
+    eligible_records_count = \
+        eligible_records_df.select(pl.len())["len"][0]
+    print(f"eligible_records_count : " +
+          f"{eligible_records_count:,.0f}")
+    samples_count = min(samples_count, eligible_records_count)
+    ctx.actual_augmentation_rate = \
+        samples_count / ctx.records_count
+    print("actual augmentation rate : " +
+          f"{ctx.actual_augmentation_rate:.1%}")
+    sampled_records_df = eligible_records_df.sample(
+        n=samples_count
+    )
+    ctx.augmented_records_df = \
+        sampled_records_df.with_columns(
+            pl.col("query")
+            .map_elements(
+                lambda query:
+                    " ".join(
+                        query.split()[
+                            :len(query.split()) // 2]),
+                return_dtype=pl.Utf8)
+            .alias("truncated_query")
+        ).select([
+            pl.col("truncated_query").alias("query"),
+            pl.lit("[]").alias("answers")
+        ])
+    print(ctx.augmented_records_df.height,
+          ctx.augmented_records_df.columns)
+    return None
+@task
+def enrich_data(_) -> None:
+    """
+    Further enrich our dataset with 'negative' records from
+    another dataset (can be general-purpose text dataset)
+    as specified by the the flow 'hf_enrich_dataset' argument.
+    """
+    """
+    Note : we here use the Hugging Face `datasets` library
+    in 'streaming' mode for records sampling.
+    """
+    hf_enrich_ds = load_dataset(
+        path=ctx.hf_enrich_dataset["repo_id"],
+        name=ctx.hf_enrich_dataset["config_name"],
+        revision=ctx.hf_enrich_dataset_dict["commit_hash"],
+        streaming=True)
+    print(hf_enrich_ds["train"])
+    samples_count = \
+        int(ctx.records_count * ctx.enrichment_rate)
+    print(f"Samplig {samples_count:,.0f} records")
+    query_attribute_handler = \
+        eval(ctx.hf_enrich_dataset["query_attribute_handler"])
+    samples_iterator = iterable_dataset_multi_buffer_sampler(
+            hf_enrich_ds["train"],
+            total_samples=samples_count,
+            attributes_selector=\
+                (lambda x:query_attribute_handler(
+                    x[ctx.hf_enrich_dataset["query_attribute"]])),
+            buffer_size=3_000,
+            num_passes=3,
+            seed=None
+        )
+    # Capitalize and add end punctuation if missing
+    start_time = time.time()
+    print("Starting sample enriching records, " +
+          "this may take some time if the source dataset " +
+          "has a complex structure..")
+    samples_list = [
+        s.capitalize() + ("" if s[-1] in ".!?" else "?")
+        for s in samples_iterator]
+    elapsed_time = time.time() - start_time
+    print(f".. sampling completed " +
+          f"({int(elapsed_time // 3_600)}h:" +
+           f"{int((elapsed_time % 3_600) // 60)}m:" +
+           f"{int(elapsed_time % 60)}s).")
+    enriched_records_df = pl.DataFrame(
+            {"query": samples_list,
+             "answers": \
+                 ["[]"] * \
+                 len(samples_list)}
+        )
+    ctx.enriched_records_df = enriched_records_df
+    return None
+@task(ui_css=UiCss(background="#FF9900", color="#111827", border="#1F2937"))
+def dataset_to_hub(_) -> None:
+    """
+    Push to hub dataset version
+    - continued pre-training dataset
+    - training and validation splits of the
+    augmented and enriched
+    supervised finetuning dataset
+    - readme with versioning info
+    """
+    #############################
+    #  case of user-provided    #
+    # documentation artifact(s) #
+    #############################
+    # note that user can provide either
+    # 'pipeline_card.py' or 'template.html'
+    # or 'dataset_readme.py'
+    # or 'dataset_readme_template.md'
+    # or 'model_readme.py'
+    # or 'model_readme_template.md'
+    # or any combination of those
+    # when specifying custom
+    # 'pipeline_card_artifacts_path'
+    if (
+        "dataset_readme_template.md" in
+            os.listdir(ctx.pipeline_card_artifacts_path)
+    ):
+        template_dir = ctx.pipeline_card_artifacts_path
+    else:
+        template_dir = os.path.dirname(
+            importlib.util.find_spec(
+                f"retrain_pipelines.pipeline_card."+
+                f"{os.getenv('retrain_pipeline_type')}"
+            ).origin)
+    print(f"template_dir : '{template_dir}'")
+    #############################
+    if "dataset_readme.py" in os.listdir(
+            ctx.pipeline_card_artifacts_path):
+        from retrain_pipelines.utils import \
+            get_get_dataset_readme_content
+        get_dataset_readme_content = \
+            get_get_dataset_readme_content(
+                ctx.pipeline_card_artifacts_path)
+    else:
+        from retrain_pipelines.pipeline_card import \
+                get_dataset_readme_content
+    #############################
+    #############################
+    #    augmented & enriched   #
+    #     finetuning dataset    #
+    #############################
+    merged_df = pl.concat([
+            # dataset
+            ctx.hf_dataset_dict["lazy_df"].select([
+                    ctx.hf_dataset["attributes"]["query_attr"],
+                    ctx.hf_dataset["attributes"]["answers_attr"]
+                ]).collect(engine=ctx.engine),
+            # truncated queries augmentation
+            ctx.augmented_records_df,
+            # enriching dataset
+            ctx.enriched_records_df
+        ]).sample(
+            # shuffling
+            fraction=1,
+            shuffle=True,
+            with_replacement=False
+        )
+    merged_df = merged_df.sample(fraction=1, shuffle=True)
+    merged_df.rechunk()
+    print(("merged_df", f"{merged_df.shape[0]:,.0F}",
+          merged_df.columns))
+    pandas_df = merged_df.to_pandas()
+    train_size = int(0.8 * len(pandas_df))
+    print(f"validation : {len(pandas_df) - train_size}")
+    sft_dataset = DatasetDict({
+        "train": Dataset.from_pandas(pandas_df[:train_size]),
+        "validation": Dataset.from_pandas(pandas_df[train_size:])
+    })
+    #############################
+    #############################
+    #   continued pre-training  #
+    #          dataset          #
+    #############################
+    struct_schema = pl.Struct([
+        pl.Field("name", pl.String),
+        pl.Field("description", pl.String),
+        pl.Field(
+            "parameters",
+            pl.String  # Use String to allow
+                       # for varying structures
+                       # (different tools indeed having
+                       #  different sets of parameters
+                       #  i.e. different parameters counts,
+                       #  datatypes and names)
+                       # so parsing must be tolerant.
+        )
+    ])
+    unique_tools_df = get_unique_tools(
+            ctx.hf_dataset_dict["lazy_df"],
+            tools_attr_name=\
+                ctx.hf_dataset["attributes"]["tools_attr"],
+            struct_schema=struct_schema
+        ).collect(engine=ctx.engine)
+    unique_tools_arrow_table = unique_tools_df.to_arrow()
+    ctx.unique_tools_dataset = \
+        Dataset(unique_tools_arrow_table)
+    print(ctx.unique_tools_dataset)
+    #############################
+    #############################
+    #        DatasetDict        #
+    #    with multiple tables   #
+    #############################
+    dataset_dict = DatasetDict({
+        "continued_pre_training": \
+            ctx.unique_tools_dataset,
+        "supervised_finetuning": sft_dataset
+    })
+    print(dataset_dict, flush=True)
+    #############################
+    #############################
+    #       dataset README      #
+    #       from template       #
+    #############################
+    commit_datetime = datetime.utcnow()
+    new_dataset_version_label = get_new_repo_minor_version(
+        repo_id=ctx.dataset_repo_id,
+        repo_type="dataset",
+        hf_token=os.getenv("HF_TOKEN", None))
+    readme_content = get_dataset_readme_content(
+        template_folder=template_dir,
+        hf_dataset_dict=ctx.hf_dataset_dict,
+        hf_enrich_dataset_dict=ctx.hf_enrich_dataset_dict,
+        dataset_dict=dataset_dict,
+        augmentation_rate=ctx.actual_augmentation_rate,
+        enrichment_rate=ctx.enrichment_rate,
+        version_label=new_dataset_version_label,
+        commit_datetime=commit_datetime,
+        pipeline_name=ctx.pipeline_name,
+        exec_id=ctx.exec_id,
+        engine=ctx.engine
+    )
+    #############################
+    dataset_commit_hash = push_dataset_version_to_hub(
+        repo_id=ctx.dataset_repo_id,
+        version_label=new_dataset_version_label,
+        timestamp_str=commit_datetime.strftime(
+            "%Y-%m-%d %H:%M:%S UTC"),
+        dataset_dict=dataset_dict,
+        dataset_readme_content=readme_content,
+        hf_token=os.getenv("HF_TOKEN", None)
+    )
+    if not dataset_commit_hash:
+        raise Exception(
+            "Failed to publish dataset version.")
+    print(f"https://huggingface.co/datasets/{ctx.dataset_repo_id}" +
+          f"/blob/{dataset_commit_hash}/README.md")
+    ctx.dataset_commit_dict = {
+        "repo_id": ctx.dataset_repo_id,
+        "commit_hash": dataset_commit_hash,
+        "version_label": new_dataset_version_label,
+        "commit_datetime": commit_datetime,
+    }
+    return None
+@task
+def continued_pre_training(_) -> None:
+    """
+    Gives the base model some additional intrinsic knowkledge
+    through continued pre-training.
+    See unsloth.ai/blog/contpretraining
+    """
+    from retrain_pipelines.model.hf_utils import \
+        plot_log_history
+    #######################################
+    # base-model and associated tokenizer #
+    #      from Hub (or local cache)      #
+    #######################################
+    ctx.max_seq_length = 2048
+    model, tokenizer = FastLanguageModel.from_pretrained(
+        model_name=ctx.hf_base_model_dict["repo_id"],
+        revision=ctx.hf_base_model_dict["commit_hash"],
+        max_seq_length=ctx.max_seq_length,
+        dtype=None,
+        load_in_4bit=False,
+        # case of a gated or private base-model
+        token=os.getenv("HF_TOKEN", None)
+    )
+    #######################################
+    #######################################
+    #   dataset prompt_template mapping   #
+    #######################################
+    tools_dataset = DatasetDict(
+        {"train": ctx.unique_tools_dataset})
+    print(tools_dataset)
+    tool_prompt_template = "tool: {}"
+    def formatting_prompts_func(tools_batch):
+        tools_batch = tools_batch["tool"]
+        outputs = []
+        for tool in tools_batch:
+            # Must add EOS_TOKEN,
+            # otherwise generation will go on forever!
+            text = tool_prompt_template.format(tool) + \
+                   tokenizer.eos_token
+            outputs.append(text)
+        return { "tools" : outputs, }
+    import dis as _dis_module
+    # silence dis.py bytecode spam (Unsloth monkey-patch side-effect)
+    _dis_module.print = lambda *a, **kw: None
+    cpt_dataset = tools_dataset["train"].map(
+        formatting_prompts_func, batched=True,)
+    del _dis_module.print
+    #######################################
+    #######################################
+    #             PEFT adapter            #
+    #      for continued pre-training     #
+    #######################################
+    model = FastLanguageModel.get_peft_model(
+        model,
+        r = 128, # any number >0 ; 8, 16, 32, 64, 128, 256
+        target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
+                          "gate_proj", "up_proj", "down_proj",
+                          # Add for continued pretraining
+                          "embed_tokens", "lm_head",],
+        lora_alpha = 32,
+        lora_dropout = 0,    # Supports any, 0 is optimized
+        bias = "none",       # Supports any, "none" is optimized
+        # True or "unsloth" for very long context
+        use_gradient_checkpointing = "unsloth",
+        use_rslora = True,   # rank-stabilized LoRA
+        loftq_config = None, # LoftQ
+        #random_state = 3407,
+    )
+    #######################################
+    #######################################
+    #             cpt_trainer             #
+    #######################################
+    if (
+        "records_cap" in ctx.cpt_training_args and
+        ctx.cpt_training_args["records_cap"] is not None and
+        isinstance(ctx.cpt_training_args["records_cap"], int)
+    ):
+        cpt_dataset = cpt_dataset.take(
+            ctx.cpt_training_args["records_cap"])
+        print(f"cpt_dataset : {cpt_dataset}")
+    train_args = UnslothTrainingArguments(
+        # https://huggingface.co/docs/transformers/main_classes/trainer#transformers.TrainingArguments.save_strategy
+        per_device_train_batch_size=2,
+        gradient_accumulation_steps=8,
+        **{k: v for k, v in ctx.cpt_training_args.items()
+                if k != "records_cap"},
+        # 2 to 10x smaller learning rate
+        # for the embedding matrices
+        learning_rate=5e-5,
+        embedding_learning_rate=1e-5,
+        fp16=not is_bfloat16_supported(),
+        bf16=is_bfloat16_supported(),
+        logging_steps=1,
+        optim="adamw_8bit",
+        weight_decay=0.01,
+        lr_scheduler_type="linear",
+        #seed=3407,
+        output_dir=os.path.join(
+            ctx.unsloth_dir, "outputs", "cpt"),
+        save_total_limit = 2,
+        report_to="tensorboard",
+        logging_dir=os.path.join(
+            ctx.sft_model_dir,
+            "runs", "cpt")
+    )
+    # silence dis.py bytecode spam (Unsloth monkey-patch side-effect)
+    _dis_module.print = lambda *a, **kw: None
+    trainer = UnslothTrainer(
+        model=model, tokenizer=tokenizer,
+        train_dataset=cpt_dataset,
+        dataset_text_field="tools",
+        max_seq_length=ctx.max_seq_length,
+        dataset_num_proc=2,
+        args=train_args,
+    )
+    del _dis_module.print
+    #######################################
+    #######################################
+    #      Show current memory stats      #
+    #######################################
+    torch.cuda.ipc_collect()
+    torch.cuda.empty_cache()
+    _ = gc.collect()
+    gpu_stats = torch.cuda.get_device_properties(0)
+    ctx.start_gpu_memory = \
+        round(torch.cuda.max_memory_reserved()
+              / 1024 / 1024 / 1024, 3)
+    ctx.max_memory = \
+        round(gpu_stats.total_memory
+              / 1024 / 1024 / 1024, 3)
+    print(f"GPU = {gpu_stats.name}. " +
+          f"Max memory = {ctx.max_memory} GB.")
+    print(f"{ctx.start_gpu_memory} GB of memory reserved.")
+    #######################################
+    ctx.cpt_traces_file_fullname = os.path.join(
+        ctx.unsloth_dir, "cpt_trainer_traces.txt")
+    logger.info(
+        "Training started. " +
+        f"Check [underline]{ctx.cpt_traces_file_fullname}[/] for live traces " +
+        "or go watch your [white bold]TensorBoard[/] charts live updates !"
+    )
+    with open(ctx.cpt_traces_file_fullname, 'w') as f:
+        with rp_redirect_stdout(f):
+            trainer_stats = trainer.train()
+    print(f"{trainer_stats.metrics['train_runtime']} " +
+          f"seconds used for CPT training " +
+          f"({round(trainer_stats.metrics['train_runtime']/60, 2)}" +
+          f" minutes).")
+    ctx.cpt_log_history = trainer.state.log_history
+    ctx.cpt_log_history_fig = \
+        plot_log_history(
+            ctx.cpt_log_history,
+            title="Continued pretraining loss"
+        )
+    del trainer
+    # logger.debug(f"Continued pretraining loss curve : {ctx.cpt_log_history}")
+    model.save_pretrained_merged(
+        save_directory=ctx.cpt_model_dir,
+        tokenizer=tokenizer,
+        save_method="lora"
+    )
+    print(f"cpt_model_dir : {ctx.cpt_model_dir}\n")
+    # vRAM & RAM cleanup
+    # (incl. force-delete all CUDA tensors in gc)
+    del model
+    del tokenizer
+    clear_gc()
+    torch.cuda.empty_cache()
+    torch.cuda.synchronize()
+    print(f"After cleanup: {torch.cuda.memory_allocated(0) / 1024**3:.2f} GB")
+    return None
+@task
+def supervised_finetuning(_) -> None:
+    """
+    Trains the model on tool-calling
+    task specialization.
+    """
+    from retrain_pipelines.model.hf_utils import \
+        plot_log_history
+    model, tokenizer = FastLanguageModel.from_pretrained(
+        model_name=ctx.cpt_model_dir,
+        max_seq_length=ctx.max_seq_length,
+        dtype=None,
+        load_in_4bit=False,
+    )
+    # !!!! bug fix BEGIN !!!!
+    # otherwise, 'embed_tokens' and 'lm_head'
+    # trained during CPT are "ignored",
+    # i.e. not saved after SFT
+    # (note that, alternatively, we could also
+    #  do this fix after sft-training and
+    #  just before saving ;
+    #  which would be equivalent to
+    #  freezing embeddings during finetuning
+    #  for better pretrained knowledge retention)
+    # @see https://www.reddit.com/r/unsloth/comments/1dtzcd6/fastlanguagemodelpatch_peft_model_changing/
+    model.model.model.embed_tokens.modules_to_save.default.to(
+        device="cuda:0",
+        dtype=torch.float32,
+        non_blocking=True)
+    model.model.model.embed_tokens.modules_to_save.default \
+        .requires_grad_(True)
+    model.model.lm_head.modules_to_save.default.to(
+        device="cuda:0",
+        dtype=torch.float32,
+        non_blocking=True)
+    model.model.lm_head.modules_to_save.default \
+        .requires_grad_(True)
+    # !!!! bug fix END !!!!
+    #######################################
+    #   dataset prompt_template mapping   #
+    #######################################
+    # download from Hub (or get from local cache)
+    queries_dataset = load_dataset(
+        path=ctx.dataset_commit_dict["repo_id"],
+        name="supervised_finetuning",
+        revision=ctx.dataset_commit_dict["commit_hash"],
+        token=os.getenv("HF_TOKEN", None))
+    print(f"HF_DATASETS_CACHE : {HF_DATASETS_CACHE}") # HF_CACHE_HOME
+    ctx.sft_prompt_template = dedent("""
+    You specialize in generating tool calls. Given a query, your task is to return a list of tool calls based on your knowledge of known tools.
+    Rules:
+    1. You can only use tools you know. Do not create new tools under any circumstances.
+    2. If a query does not match any known tool, return an empty list ([]).
+    3. If information is missing to use a known tool, do not attempt to use it.
+    4. Your response must always be a valid JSON array, and nothing else.
+    Be precise and do not guess.
+    # query:
+        {}
+    # response:
+        {}
+    """).strip()
+    tokenizer.chat_template = ctx.sft_prompt_template
+    EOS_TOKEN = tokenizer.eos_token
+    def formatting_prompts_func(records):
+        query = records["query"]
+        tools  = records["answers"]
+        outputs = []
+        for query, tools in zip(query, tools):
+            # Must add EOS_TOKEN,
+            # otherwise your generation will go on forever
+            text = ctx.sft_prompt_template.format(query, tools) \
+                   + EOS_TOKEN
+            outputs.append(text)
+        return { "text" : outputs, }
+    import dis as _dis_module
+    # silence dis.py bytecode spam (Unsloth monkey-patch side-effect)
+    _dis_module.print = lambda *a, **kw: None
+    sft_train_dataset = queries_dataset["train"].map(
+        formatting_prompts_func, batched=True)
+    sft_valid_dataset = queries_dataset["validation"].map(
+        formatting_prompts_func, batched=True,)
+    del _dis_module.print
+    #######################################
+    #######################################
+    #             PEFT adapter            #
+    #      for supervised finetuning      #
+    #######################################
+    # for cases where CPT has been merged into overall model
+    # otherwize, keep on training current LoRa adapter
+    # model = FastLanguageModel.get_peft_model(
+        # model,
+        # r = 128, # any number >0 ; 8, 16, 32, 64, 128, 256
+        # target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
+                          # "gate_proj", "up_proj", "down_proj"],
+        # lora_alpha = 32,
+        # lora_dropout = 0, # Supports any, but = 0 is optimized
+        # bias = "none",    # Supports any, but = "none" is optimized
+        # # True or "unsloth" for very long context
+        # use_gradient_checkpointing = "unsloth",
+        # random_state = 3407,
+        # use_rslora = True,   # rank stabilized LoRA
+        # loftq_config = None, # LoftQ
+    # )
+    #######################################
+    #######################################
+    #             sft_trainer             #
+    #######################################
+    split = sft_train_dataset.train_test_split(
+        test_size=1000,
+        #seed=42
+    )
+    train_dataset = split['train']
+    eval_dataset = split['test']
+    if (
+        "records_cap" in ctx.sft_training_args and
+        ctx.sft_training_args["records_cap"] is not None and
+        isinstance(ctx.sft_training_args["records_cap"], int)
+    ):
+        train_dataset = train_dataset.take(
+            ctx.sft_training_args["records_cap"])
+        eval_dataset = eval_dataset.take(
+            ctx.sft_training_args["records_cap"])
+        print(f"train_dataset : {train_dataset}")
+        print(f"eval_dataset :  {eval_dataset}")
+    train_args = UnslothTrainingArguments(
+        per_device_train_batch_size=2,
+        gradient_accumulation_steps=8,
+        **{k: v for k, v in ctx.sft_training_args.items()
+                if k != "records_cap"},
+        per_device_eval_batch_size=2,
+        eval_steps=200,
+        eval_strategy="steps",
+        do_eval=True,
+        learning_rate=5e-5,
+        # embedding_learning_rate=1e-5, # Optionally here
+        fp16=not is_bfloat16_supported(),
+        bf16=is_bfloat16_supported(),
+        optim="adamw_8bit",
+        weight_decay=0.00,
+        lr_scheduler_type="linear",
+        #seed=3407,
+        output_dir=os.path.join(
+            ctx.unsloth_dir, "outputs", "sft"),
+        save_total_limit=2,
+        disable_tqdm=True,
+        logging_steps=1,
+        report_to="tensorboard",
+        logging_dir=os.path.join(
+            ctx.sft_model_dir,
+            "runs", "sft")
+    )
+    # silence dis.py bytecode spam (Unsloth monkey-patch side-effect)
+    _dis_module.print = lambda *a, **kw: None
+    trainer = UnslothTrainer(
+        model=model, tokenizer=tokenizer,
+        train_dataset=train_dataset,
+        dataset_text_field="text",
+        eval_dataset=eval_dataset,
+        max_seq_length=ctx.max_seq_length,
+        dataset_num_proc=8,
+        args=train_args
+    )
+    del _dis_module.print
+    trainer.can_return_loss = True
+    #######################################
+    #######################################
+    #      Show current memory stats      #
+    #######################################
+    torch.cuda.ipc_collect()
+    torch.cuda.empty_cache()
+    _ = gc.collect()
+    used_memory = \
+        round(torch.cuda.max_memory_reserved()
+              /1024/1024/1024, 3)
+    used_memory_for_lora = \
+        round(used_memory-ctx.start_gpu_memory, 3)
+    used_percentage = \
+        round(used_memory/ctx.max_memory*100, 3)
+    lora_percentage = \
+        round(used_memory_for_lora/ctx.max_memory*100,
+              3)
+    print(f"Peak reserved memory = " +
+          f"{used_memory} GB.")
+    print(f"Peak reserved memory for " +
+          f"training = {used_memory_for_lora} " +
+          f"GB.")
+    print(f"Peak reserved memory % of " +
+          f"max memory = {used_percentage} %.")
+    print(f"Peak reserved memory for SFT training " +
+          f"% of max memory = {lora_percentage} %.")
+    #######################################
+    ctx.sft_traces_file_fullname = os.path.join(
+        ctx.unsloth_dir, "sft_trainer_traces.txt")
+    logger.info(
+        "Training started. " +
+        f"Check [underline]{ctx.sft_traces_file_fullname}[/] for live traces " +
+        "or go watch your [white bold]TensorBoard[/] charts live updates !"
+    )
+    with open(ctx.sft_traces_file_fullname, 'w') as f:
+        with rp_redirect_stdout(f):
+            trainer_stats = trainer.train()
+    print(f"{trainer_stats.metrics['train_runtime']} " +
+          f"seconds used for training " +
+          f"({round(trainer_stats.metrics['train_runtime']/60, 2)}" +
+          f" minutes).")
+    ctx.sft_log_history = trainer.state.log_history
+    ctx.sft_log_history_fig = \
+        plot_log_history(
+            ctx.sft_log_history,
+            title="Supervised finetuning loss"
+        )
+    del trainer
+    model.save_pretrained_merged(
+        ctx.sft_model_dir, tokenizer,
+        save_method = "lora"
+    )
+    print(f"sft_model_dir : {ctx.sft_model_dir}\n")
+    # vRAM & RAM cleanup
+    # (incl. force-delete all CUDA tensors in gc)
+    del model
+    del tokenizer
+    clear_gc()
+    torch.cuda.empty_cache()
+    torch.cuda.synchronize()
+    print(f"After cleanup: {torch.cuda.memory_allocated(0) / 1024**3:.2f} GB")
+    return None
+@task
+def evaluate_model(_) -> None:
+    """
+    Batch inference on the SFT validation dataset.
+    """
+    from retrain_pipelines.model import \
+        infer_validation, compute_counts_n_metrics, \
+        plot_validation_completions
+    ######################################################
+    #              loading trained adapter               #
+    ######################################################
+    # Unsloth [and hf transformers before it]            #
+    # (if loading both model & tokenizer at once         #
+    # same as we did in prior tasks, but now             #
+    # with tokenizer.chat_template being set             #
+    # in tokenizer.config) is forcing on us some kind of #
+    # chat_template format hard-requirements.            #
+    ######################################################
+    # load base from cache
+    # (with base tokenizer, which we ignore)
+    model, _ = FastLanguageModel.from_pretrained(
+        model_name=ctx.hf_base_model_dict["repo_id"],
+        revision=ctx.hf_base_model_dict["commit_hash"],
+        max_seq_length=ctx.max_seq_length,
+        dtype=None,
+        load_in_4bit=False,
+        # case of a gated or private base-model
+        token=os.getenv("HF_TOKEN", None)
+    )
+    model = FastLanguageModel.for_inference(model)
+    # load our CPT+SFT trained & locally-saved adapter
+    model.load_adapter(peft_model_id=ctx.sft_model_dir)
+    # Separately load our (potentially trained &)
+    # locally-saved adapter-tokenizer
+    # (loading it below via HF and not Unsloth)
+    tokenizer = AutoTokenizer.from_pretrained(
+        pretrained_model_name_or_path=ctx.sft_model_dir
+    )
+    ######################################################
+    ######################################################
+    #                 validation dataset                 #
+    ######################################################
+    # download from Hub (or get from local cache)
+    queries_dataset = load_dataset(
+        path=ctx.dataset_commit_dict["repo_id"],
+        name="supervised_finetuning",
+        revision=ctx.dataset_commit_dict["commit_hash"],
+        token=os.getenv("HF_TOKEN", None))
+    if (
+        "records_cap" in ctx.sft_training_args and
+        ctx.sft_training_args["records_cap"] is not None and
+        isinstance(ctx.sft_training_args["records_cap"], int)
+    ):
+        validation_data = queries_dataset["validation"].take(
+            ctx.sft_training_args["records_cap"])
+    else:
+        validation_data = queries_dataset["validation"]
+    print(validation_data, flush=True)
+    ######################################################
+    ctx.max_new_tokens = 400
+    start_time = time.time()
+    validation_results = infer_validation(
+        tokenizer=tokenizer,
+        model=model,
+        validation_data=validation_data,
+        prompt_template=tokenizer.chat_template,
+        batch_size=32, # 64,
+        queries_attr_name=\
+            ctx.hf_dataset["attributes"]["query_attr"],
+        answers_attr_name=\
+            ctx.hf_dataset["attributes"]["answers_attr"],
+        max_new_tokens=ctx.max_new_tokens,
+        device="cuda"
+    )
+    print("infer_validation -   Elapsed time: " +
+          f"{(time.time() - start_time):.2f} seconds")
+    ctx.validation_results = validation_results #  <= to artifacts store
+    eval_df  = pl.LazyFrame(validation_results)
+    records = eval_df.with_columns(
+        (pl.col("answer") == pl.col("completion")) \
+            .alias("is_ground_truth_identical")
+    ).collect() #engine=ctx.engine)
+    print("perfect characters-match accuracy : " +
+          str(records['is_ground_truth_identical'].mean()))
+    eval_metrics_df = compute_counts_n_metrics(
+        eval_df, is_format_fault_tolerant=True)
+    overall_metrics_df = eval_metrics_df.select([
+            pl.col("precision").mean(),
+            pl.col("recall").mean(),
+            pl.col("f1").mean(),
+            pl.col("jaccard").mean()
+        ]).collect() #engine=ctx.engine)
+    ctx.perf_metrics = overall_metrics_df.row(0, named=True)
+    print(ctx.perf_metrics)
+    ctx.validation_completions_fig = \
+        plot_validation_completions(
+            eval_metrics_df, engine=ctx.engine)
+    # vRAM & RAM cleanup
+    # (incl. force-delete all CUDA tensors in gc)
+    del model
+    del tokenizer
+    clear_gc()
+    torch.cuda.empty_cache()
+    torch.cuda.synchronize()
+    print(f"After cleanup: {torch.cuda.memory_allocated(0) / 1024**3:.2f} GB")
+    return None
+@task
+def model_version_blessing(_) -> None:
+    """
+    Comparing newly-retrained model version
+    against best-performing predecessor.
+    """
+    """
+    Note: for Hugging Face integrated pipelines,
+    we compare against lastest commit of main branch
+    of the model repository there.
+    When it comes to local "mf_run_id" of the pipeline run
+    having generated that best prior model version
+    (retrieved from model card metadata from HF yaml section),
+    we check against records of the herein ML-framework instance,
+    as "prior best version" of the model here beign retrained
+    may have been originated from another one
+    than the one executing the current retraining
+    (in which case, we simply don't includ a "local" hyperlink
+    in the model version pipeline_cards that will be
+    produced later in the herein pipeline run).
+    """
+    from retrain_pipelines.model.hf_utils import \
+        current_blessed_model_version_dict
+    main_perf_metric_name = "jaccard"
+    current_blessed_version_dict = \
+        current_blessed_model_version_dict(
+            repo_id=ctx.model_repo_id,
+            hf_token=os.getenv("HF_TOKEN", None)
+        )
+    print("current_blessed_version_dict : " +
+          str(current_blessed_version_dict))
+    if current_blessed_version_dict is None:
+        print("case 'no prior blessed model version found"
+              " => blessing.'")
+        ctx.model_version_blessed = True
+    elif (
+        main_perf_metric_name in
+            current_blessed_version_dict["perf_metrics"]
+    ):
+        current_blessed_exec_id = \
+            current_blessed_version_dict["exec_id"]
+        print(f"current_blessed_exec_id : {current_blessed_exec_id}")
+        current_blessed_metric_value = \
+            current_blessed_version_dict[
+                "perf_metrics"][main_perf_metric_name]
+        ctx.model_version_blessed = (
+            ctx.perf_metrics[main_perf_metric_name] >=
+            current_blessed_metric_value
+        )
+        # ctx.model_version_blessed = False ### DEBUG - DELETE ###
+        if not ctx.model_version_blessed:
+            ctx.current_blessed_version_dict = \
+                current_blessed_version_dict
+            # may have failed after the "pipeline_card" task,
+            # so we do not filter on success
+            for execution in ExecutionsIterator(
+                exec_name=ctx.pipeline_name,
+                page_size=10
+            ):
+                if str(execution.id) == current_blessed_exec_id:
+                    # Has the execution seen task "pipeline_card" which
+                    # completed successfully
+                    # ("execution" has generated a custom pipeline-card artifact) ?
+                    # If not, hyperlink generation will later fail.
+                    run_has_custom_card_artifact = (len([
+                        t for t in execution.get_tasks_with_name(
+                                        task_type_name="pipeline_card")
+                        if t.end_timestamp and t.success
+                    ]) == 1)
+                    if not run_has_custom_card_artifact:
+                        print(
+                            f"Execution #{current_blessed_exec_id} " +
+                            "Doesn't seem to have successfully " +
+                            "generated a pipeline-card artifact.",
+                            file=sys.stderr, flush=True)
+                    else:
+                        # further filtering on successful executions that are
+                        # retraining of a prior version of the same model
+                        # (to minimize the risk that this was obtained
+                        #  on another DAG-engine instance)
+                        if (
+                            execution.get_attr("model_version_blessed") and
+                            execution.get_attr("model_repo_id") or "" == \
+                                ctx.model_repo_id
+                        ):
+                            ctx.current_blessed_exec = execution
+                    break
+            if not ctx.current_blessed_exec:
+                logger.warning(
+                    "Couldn't find blessed execution " +
+                    f"{current_blessed_exec_id} !\n" +
+                    "It seems that prior blessed execution was " +
+                    "executed on another DAG-engine instance.")
+            else:
+                logger.debug(
+                    f"ctx.current_blessed_exec : {ctx.current_blessed_exec}")
+        print("new : " +
+                str(ctx.perf_metrics[main_perf_metric_name]) +
+              " - previous best : " +
+                str(current_blessed_metric_value) +
+              " - model_version_blessing : " +
+                str(ctx.model_version_blessed))
+    else:
+        raise Exception(
+            "Performance metric '" +
+            main_perf_metric_name +
+            "' can't be found in eval results " +
+            "from blessed execution " +
+            str(current_blessed_version_dict[
+                "exec_id"]) + " !")
+    return None
+@task(ui_css=UiCss(background="#FF9900", color="#111827", border="#1F2937"))
+def model_to_hub(_) -> None:
+    """
+    Push to hub model version, including
+    readme with versioning info.
+    """
+    #############################
+    #  case of user-provided    #
+    # documentation artifact(s) #
+    #############################
+    # note that user can provide either
+    # 'pipeline_card.py' or 'template.html'
+    # or 'dataset_readme.py'
+    # or 'dataset_readme_template.md'
+    # or 'model_readme.py'
+    # or 'model_readme_template.md'
+    # or any combination of those
+    # when specifying custom
+    # 'pipeline_card_artifacts_path'
+    if (
+        "model_readme_template.md" in
+            os.listdir(ctx.pipeline_card_artifacts_path)
+    ):
+        template_dir = ctx.pipeline_card_artifacts_path
+    else:
+        template_dir = os.path.dirname(
+            importlib.util.find_spec(
+                f"retrain_pipelines.pipeline_card."+
+                f"{os.getenv('retrain_pipeline_type')}"
+            ).origin)
+    print(f"template_dir : '{template_dir}'")
+    #############################
+    if "model_readme.py" in os.listdir(
+            ctx.pipeline_card_artifacts_path):
+        from retrain_pipelines.utils import \
+            get_get_model_readme_content
+        get_model_readme_content = \
+            get_get_model_readme_content(
+                ctx.pipeline_card_artifacts_path)
+    else:
+        from retrain_pipelines.pipeline_card import \
+                get_model_readme_content
+    #############################
+    from retrain_pipelines.model.hf_utils import \
+        push_model_version_to_hub
+    #############################
+    #        model README       #
+    #       from template       #
+    #############################
+    commit_datetime = datetime.utcnow()
+    new_model_version_label = get_new_repo_minor_version(
+        repo_id=ctx.model_repo_id,
+        repo_type="model",
+        hf_token=os.getenv("HF_TOKEN", None))
+    readme_content = get_model_readme_content(
+        template_folder=template_dir,
+        model_repo_id=ctx.model_repo_id,
+        base_model_dict=ctx.hf_base_model_dict,
+        training_dataset_dict=ctx.dataset_commit_dict,
+        version_label=new_model_version_label,
+        commit_datetime=commit_datetime,
+        perf_metrics=ctx.perf_metrics,
+        pipeline_name=ctx.pipeline_name,
+        exec_id=ctx.exec_id
+    )
+    #############################
+    print("Pushing model version to HF hub " +
+          ("(blessed). " if ctx.model_version_blessed
+           else "(not blessed). ") +
+          "May take a while..",
+          flush=True)
+    model_commit_hash = push_model_version_to_hub(
+        repo_id=ctx.model_repo_id,
+        model_version_blessed=\
+            ctx.model_version_blessed,
+        version_label=new_model_version_label,
+        timestamp_str=commit_datetime.strftime(
+            "%Y-%m-%d %H:%M:%S UTC"),
+        model_dir=ctx.sft_model_dir,
+        model_readme_content=readme_content,
+        hf_token=os.getenv("HF_TOKEN", None)
+    )
+    if not model_commit_hash:
+        raise Exception(
+            "Failed to publish model version.")
+    print("Push of model version to HF hub completed.",
+          flush=True)
+    print(f"https://huggingface.co/{ctx.model_repo_id}" +
+          f"/blob/{model_commit_hash}/README.md")
+    ctx.model_commit_dict = {
+        "repo_id": ctx.model_repo_id,
+        "commit_hash": model_commit_hash,
+        "version_label": new_model_version_label,
+        "commit_datetime": commit_datetime,
+    }
+    return None
+@task
+def infra_validator(_) -> None:
+    """
+    If the trained model version is blessed,
+    validate serving.
+    """
+    """
+    Note that using isolated virtual env
+    (using @conda task decorator)
+    is advisable to not embark the whole
+    pipeline dependencies into the local server.
+    We don't for educational purpose,
+    keep things "simple" to grasp
+    as well as to avoid forcing conda
+    (for instance miniconda) as
+    a virtual environment management mean
+    to the user.
+    """
+    """
+    Note : We load base model from HF-cache
+    (mounted as /huggingface_hub_cache
+    docker volume) and adapter from local dir
+    (mounted as /FuncCallAdater docker volume.
+    """
+    ctx.local_serve_is_ready = LocalServeReadinessEnum.NOT_APPLICABLE
+    if ctx.model_version_blessed:
+        from retrain_pipelines.utils.docker import \
+            env_has_docker
+        if env_has_docker():
+            model_module_dir = \
+                os.path.dirname(
+                    importlib.util.find_spec(
+                        "retrain_pipelines.model." +
+                        os.getenv('retrain_pipeline_type')
+                    ).origin)
+            # server & data-model & server-config modules artifacts
+            files_to_copy = [
+                "litserve_server.py",
+                "litserve_datamodel.py",
+                "litserve_serverconfig.py",
+                ".dockerignore" # docker context loading
+                                # at image-build time,
+                                # exclude model weights
+            ]
+            for filename in files_to_copy:
+                shutil.copy(
+                    os.path.join(model_module_dir, "litserve",
+                                 filename),
+                    os.path.join(ctx.serving_artifacts_local_folder,
+                                 filename)
+                )
+            # save dependencies as artifact
+            create_requirements(ctx.serving_artifacts_local_folder,
+                                exclude=["numpy",  # version conflict
+                                                   # quick fix
+                                         "cudf-polars-.*", "cuda-python",
+                                         "nvidia-.*", "(py)?libcudf-.*",
+                                         "nvtx", "rmm-.*", "litserve",
+                                         "protobuf", "grpc.*",
+                                         "tensorboard",
+                                         ".*retrain-pipelines.*"]
+            )
+            # server config yaml
+            env = Environment(loader=FileSystemLoader(
+                os.path.join(model_module_dir, "litserve")))
+            template = env.get_template(
+                "litserve_serverconfig_template.yaml")
+            server_config_data = {
+                "port": "8000",
+                "max_seq_length": ctx.max_seq_length,
+                "max_new_token": ctx.max_new_tokens,
+                "base_model": {
+                    "repo_id": ctx.hf_base_model_dict["repo_id"],
+                    "revision": ctx.hf_base_model_dict["commit_hash"]
+                },
+                "adapters": [
+                    {
+                        "name": "func_caller",
+                        "path": "/FuncCallAdapter"
+                    }
+                ]
+            }
+            server_config_yaml = template.render(server_config_data)
+            print(server_config_yaml)
+            with open(os.path.join(
+                ctx.serving_artifacts_local_folder,
+                "litserve_serverconfig.yaml"), 'w'
+            ) as output_file:
+                output_file.write(server_config_yaml)
+            # Dockerfile
+            env = Environment(loader=FileSystemLoader(
+                os.path.join(model_module_dir)))
+            template = env.get_template(
+                "Dockerfile.litserve_template")
+            # Change CUDA version here from available list
+            # @see https://hub.docker.com/r/nvidia/cuda/tags
+            dockerfile_content = template.render(
+                {"cuda_version": "12.0.0"})
+            with open(os.path.join(
+                ctx.serving_artifacts_local_folder,
+                "Dockerfile.litserve"), 'w'
+            ) as output_file:
+                output_file.write(dockerfile_content)
+            os.environ["no_proxy"] = "localhost,127.0.0.1,0.0.0.0"
+            ############################################
+            #   actually deploy the inference service  #
+            ############################################
+            start_time = time.time()
+            from retrain_pipelines.utils.docker import \
+                build_and_run_docker, print_container_log_tail, \
+                cleanup_docker
+            from retrain_pipelines.model.litserve import \
+                endpoint_started, endpoint_is_ready
+            ctx.port = 8765
+            HF_HUB_CACHE = os.path.realpath(os.path.expanduser(
+                os.getenv(
+                    "HF_HUB_CACHE",
+                    os.path.join(os.getenv("HF_HOME",
+                                           "~/.cache/huggingface"),
+                                 "hub")
+                )))
+            print(f"HF_HUB_CACHE : {HF_HUB_CACHE}")
+            image_name = container_name = "litserve-model"
+            serving_container = build_and_run_docker(
+                image_name=image_name, image_tag="1.0",
+                build_path=ctx.serving_artifacts_local_folder,
+                dockerfile="Dockerfile.litserve",
+                ports_publish_dict={'8000/tcp': ctx.port},
+                env_vars_dict={
+                    "HF_HUB_CACHE": "/huggingface_hub_cache",
+                    "HF_TOKEN": os.getenv("HF_TOKEN")
+                },
+                volumes_dict={
+                    ctx.sft_model_dir:
+                        {"bind": "/FuncCallAdapter",
+                         "mode": "ro"},
+                    HF_HUB_CACHE:
+                        {"bind": "/huggingface_hub_cache",
+                         "mode": "ro"}
+                }
+            )
+            if not serving_container:
+                print("failed spinning the LitServe container",
+                      file=sys.stderr)
+                ctx.local_serve_is_ready = \
+                    LocalServeReadinessEnum.FAILURE
+                try:
+                    cleanup_docker(
+                        container_name=container_name,
+                        image_name=f"{image_name}:1.0",
+                        no_pruning=True # for intermediate layers recycling
+                                        # (during later re-runs)
+                                        # to avoid long rebuild time
+                                        # of exactly the same.
+                    )
+                except Exception as cleanup_ex:
+                    # fail silently
+                    pass
+            else:
+                print("Awaiting endpoint launch..")
+                start_time = time.time()
+                if not endpoint_started(
+                    container_name, port=ctx.port, timeout=10*60
+                ):
+                    print(
+                        f"The endpoint '{container_name}' " +
+                        f"did not start.")
+                    ctx.local_serve_is_ready = \
+                        LocalServeReadinessEnum.FAILURE
+                # health check on the spun-up endpoint
+                elif endpoint_is_ready(port=ctx.port):
+                    ctx.local_serve_is_ready = \
+                        LocalServeReadinessEnum.SUCCESS
+            elapsed_time = time.time() - start_time
+            print("deploy_local -   Elapsed time: " +
+                  f"{elapsed_time:.2f} seconds")
+            ############################################
+        else:
+            # env doesn't have docker
+            ctx.local_serve_is_ready = \
+                LocalServeReadinessEnum.FAILURE_NO_DOCKER
+        if LocalServeReadinessEnum.SUCCESS == ctx.local_serve_is_ready:
+            from retrain_pipelines.model.litserve.litserve_datamodel \
+                import Response
+            import requests
+            url = f"http://localhost:{ctx.port}/predict"
+            headers = {"accept": "application/x-www-form-urlencoded"}
+            try:
+                start_time = time.time()
+                data = {
+                    "adapter_name": "func_caller",
+                    "queries_list": '["Hello.", "Is 49 a perfect square?"]'
+                }
+                print(f"inference test - data: {data}")
+                response = requests.post(url, headers=headers, data=data)
+                parsed_response = Response(**{"output": response.json()})
+                elapsed_time = time.time() - start_time
+                print("parsed_response ('func_caller' adapter ON) :" +
+                      str(parsed_response) +
+                      f"\t-\tElapsed time: {elapsed_time:.2f} seconds")
+                start_time = time.time()
+                data = {
+                    "queries_list": '["Hello.", "Is 49 a perfect square?"]'
+                }
+                print(f"inference test - data: {data}")
+                response = requests.post(url, headers=headers, data=data)
+                parsed_response = Response(**{"output": response.json()})
+                elapsed_time = time.time() - start_time
+                print(f"parsed_response (no adapter) : {parsed_response}" +
+                      f"\t-\tElapsed time: {elapsed_time:.2f} seconds")
+            except Exception as ex:
+                print(ex, file=sys.stderr)
+                traceback.print_tb(ex.__traceback__, file=sys.stderr)
+                ctx.local_serve_is_ready = \
+                    LocalServeReadinessEnum.FAILURE
+                pass
+        try:
+            cleanup_docker(
+                container_name=container_name,
+                image_name=f"{image_name}:1.0",
+                no_pruning=True # for intermediate layers recycling
+                                # (during later re-runs)
+                                # to avoid long rebuild time
+                                # of exactly the same.
+            )
+        except Exception as cleanup_ex:
+            # fail silently
+            pass
+    else:
+        logger.info("model-version not blessed - skipping")
+    return None
+@task
+def pipeline_card(_, task_id: int) -> None:
+    #############################
+    #   case of user-provided   #
+    # documentation artifact(s) #
+    #############################
+    # note that user can provide either
+    # 'pipeline_card.py' or 'template.html'
+    # or 'dataset_readme.py'
+    # or 'dataset_readme_template.md'
+    # or 'model_readme.py'
+    # or 'model_readme_template.md'
+    # or any combination of those
+    # when specifying custom
+    # 'pipeline_card_artifacts_path'
+    if "template.html" in os.listdir(
+                            ctx.pipeline_card_artifacts_path
+    ):
+        template_dir = ctx.pipeline_card_artifacts_path
+    else:
+        template_dir = os.path.dirname(
+            importlib.util.find_spec(
+                f"retrain_pipelines.pipeline_card."+
+                f"{os.getenv('retrain_pipeline_type')}"
+            ).origin)
+    #############################
+    if "pipeline_card.py" in os.listdir(
+                                ctx.pipeline_card_artifacts_path
+    ):
+        from retrain_pipelines.utils import get_get_html
+        get_html = \
+            get_get_html(ctx.pipeline_card_artifacts_path)
+    else:
+        from retrain_pipelines.pipeline_card import \
+                get_html
+    from retrain_pipelines.dag_engine.renderer import dag_svg
+    #############################
+    #############################
+    ##    html "custom" card   ##
+    #############################
+    dt = datetime.now(tz=timezone.utc)
+    formatted_dt = dt.strftime("%A %b %d %Y %I:%M:%S %p %Z")
+    task_obj_python_cmd = f"sdk.Task({task_id})"
+    executions_count = ExecutionsIterator(
+        exec_name=ctx.pipeline_name).length()
+    params={
+        'template_dir': template_dir,
+        'title': ctx.pipeline_name,
+        "subtitle": f"(Pipeline execution # {executions_count}," + \
+                    f" exec_id: {str(ctx.exec_id)}  -  {formatted_dt})",
+        # blessed status / current_blessed version
+        'model_version_blessed': ctx.model_version_blessed,
+        'current_blessed_version_label': (
+            ctx.current_blessed_version_dict["version_label"]
+            if ctx.current_blessed_version_dict
+            else None
+        ),
+        'current_blessed_commit_datetime': (
+            ctx.current_blessed_version_dict["commit_datetime"]
+            if ctx.current_blessed_version_dict
+            else None
+        ),
+        'current_blessed_model_commit_hash': (
+            ctx.current_blessed_version_dict["commit_hash"]
+            if ctx.current_blessed_version_dict
+            else None
+        ),
+        'current_blessed_run': ctx.current_blessed_exec,
+        'LocalServeReadinessEnum': LocalServeReadinessEnum,
+        'local_serve_is_ready': ctx.local_serve_is_ready,
+        # EDA
+        'main_dataset_repo_id': ctx.hf_dataset['repo_id'],
+        'main_dataset_commit_hash': ctx.hf_dataset_dict['commit_hash'],
+        'main_dataset_commit_datetime': \
+            ctx.hf_dataset_dict['commit_datetime'],
+        'records_count': ctx.records_count,
+        'data_schema': ctx.data_schema,
+        'answers_tools_count_fig': ctx.answers_tools_count_fig,
+        'words_count_fig': ctx.words_count_fig,
+        # model training
+        'dataset_repo_id': ctx.dataset_repo_id,
+        'dataset_version_label': ctx.dataset_commit_dict["version_label"],
+        'dataset_commit_datetime': ctx.dataset_commit_dict["commit_datetime"],
+        'dataset_commit_hash': ctx.dataset_commit_dict["commit_hash"],
+        'dataset_augmentation_rate': ctx.actual_augmentation_rate,
+        'dataset_enrichment_rate': ctx.enrichment_rate,
+        # trained model version
+        'model_repo_id': ctx.model_repo_id,
+        'model_version_label': ctx.model_commit_dict["version_label"],
+        'model_commit_datetime': ctx.model_commit_dict["commit_datetime"],
+        'model_commit_hash': ctx.model_commit_dict["commit_hash"],
+        'cpt_log_history_fig': ctx.cpt_log_history_fig,
+        'sft_log_history_fig': ctx.sft_log_history_fig,
+        'validation_completions_fig': ctx.validation_completions_fig,
+        'hf_base_model_dict': ctx.hf_base_model_dict,
+        'pipeline_parameters_dict': {"cpt": ctx.cpt_training_args,
+                                     "sft": ctx.sft_training_args},
+        'metrics_dict': ctx.perf_metrics,
+        'task_obj_python_cmd': task_obj_python_cmd,
+        'dag_svg': dag_svg(execution_id=ctx.exec_id)
+    }
+    html = get_html(params)
+    filename = os.path.join(
+        os.environ["RP_ARTIFACTS_STORE"],
+        ctx.pipeline_name, str(ctx.exec_id),
+        "pipeline_card.html"
+    )
+    os.makedirs(os.path.dirname(filename), exist_ok=True)
+    with open(filename, "w", encoding="utf-8") as file:
+        file.write(html)
+    logger.debug(
+        "pipeline_card - " +
+        f"[bold]pipeline_card_file_fullname : {filename}[/]")
+    ctx.pipeline_card_file_fullname = filename
+    #############################
+    return None
+@task(ui_css=UiCss(background="#FF9900", color="#111827", border="#1F2937"))
+def pipeline_to_hub(_) -> None:
+    """
+    publish versioned source-code and pipeline-card
+    for ths run on the Hugging Face Hub.
+    """
+    model_commit_datetime = \
+        ctx.model_commit_dict["commit_datetime"]
+    timestamp_str = \
+        "{:%Y%m%d_%H%M%S}".format(model_commit_datetime) + \
+        "{:03d}".format(model_commit_datetime.microsecond//1000) + \
+        "_UTC"
+    subfolder_name = \
+        "v" + ctx.model_commit_dict["version_label"] + \
+        "_" + timestamp_str
+    commit_datetime = datetime.utcnow()
+    ###############################
+    #         source-code         #
+    ###############################
+    # We upload only herein file  #
+    # plus user-provided versions #
+    # of the customizable ones    #
+    # (if any).                   #
+    ###############################
+    custom_source_files = [os.path.abspath(__file__)]
+    if (
+        ctx.pipeline_card_artifacts_path != \
+        ctx.params_definitions["pipeline_card_artifacts_path"].default
+    ):
+        candidate_source_files = [
+            "pipeline_card.py",
+            "template.html",
+            "dataset_readme.py",
+            "dataset_readme_template.md",
+            "model_readme.py",
+            "model_readme_template.md"
+        ]
+        for candidate_source_file in candidate_source_files:
+            file_fullpath = os.path.join(
+                ctx.pipeline_card_artifacts_path,
+                candidate_source_file)
+            if os.path.exists(file_fullpath):
+                custom_source_files.append(file_fullpath)
+    source_code_commit_hash = \
+        push_files_to_hub_repo_branch(
+            repo_id=ctx.model_repo_id,
+            branch_name="retrain-pipelines_source-code",
+            file_fullnames=custom_source_files,
+            include_requirements_txt=True,
+            path_in_repo=subfolder_name,
+            commit_message=\
+                "source-code for model version " + \
+                subfolder_name + \
+                f"- retrain-pipelines {__version__}",
+            repo_type="model",
+            hf_token=os.getenv("HF_TOKEN", None)
+        )
+    print(source_code_commit_hash)
+    ctx.source_code_commit_dict = {
+        "repo_id": ctx.model_repo_id,
+        "branch_name": "retrain-pipelines_source-code",
+        "commit_datetime": commit_datetime,
+        "commit_hash": source_code_commit_hash
+    }
+    ###############################
+    ###############################
+    #        pipeline-card        #
+    ###############################
+    pipeline_card_commit_hash = \
+        push_files_to_hub_repo_branch(
+            repo_id=ctx.model_repo_id,
+            branch_name="retrain-pipelines_pipeline-card",
+            file_fullnames=[ctx.pipeline_card_file_fullname],
+            path_in_repo=subfolder_name,
+            commit_message=\
+                "pipeline-card for model version " + \
+                subfolder_name + \
+                f"- retrain-pipelines {__version__}",
+            repo_type="model",
+            hf_token=os.getenv("HF_TOKEN", None)
+        )
+    print(pipeline_card_commit_hash)
+    ctx.pipeline_card_commit_dict = {
+        "repo_id": ctx.model_repo_id,
+        "branch_name": "retrain-pipelines_pipeline-card",
+        "commit_datetime": commit_datetime,
+        "commit_hash": pipeline_card_commit_hash
+    }
+    ###############################
+    return None
+@task
+def deploy(_):
+    """
+    placeholder for the serving SDK "deploy" call
+    (on the target production platform).
+    consider including the portable pipelione-card
+    itself to the inference service endpoint !
+    """
+    if ctx.model_version_blessed and (ctx.local_serve_is_ready == 1):
+        pass # your code here
+    return None
+@task
+def load_test(_):
+    """
+    placeholder
+    """
+    if ctx.model_version_blessed and (ctx.local_serve_is_ready == 1):
+        pass # your code here
+    return None
+@task
+def end(_):
+    pass
+#--- retraining-pipeline params & DAG ---------------------------------------------------
+@dag(ui_css=UiCss(color="#FFDD00", background="#7AD4FF", border="#C28E00"))
+def retrain_pipeline():
+    """
+    Retraining pipeline with SFT & CPT. Small LLM with pluggable adapter specialized in tool-calling from intrinsic knowledge bank of tools and not from extended context. Model-version blessing. Serving via a custom LitServe toy-server.
+    """
+    # @see https://github.com/unslothai/unsloth/wiki
+    #--- flow parameters -------------------------------------------------------
+    RETRAIN_PIPELINE_TYPE = "mf_unsloth_func_call_litserve"
+    # best way to share the config across subprocesses
+    os.environ["retrain_pipeline_type"] = RETRAIN_PIPELINE_TYPE
+    hf_dataset = DagParam(
+        description="dict with 'repo_id' and 'commit_hash' keys. " + \
+                    "if 'commit_hash is None, falls back to latest version " +\
+                    "of the dataset available in parquet format.\n" +
+                    "Note that there are 3 required 'attributes' of type " + \
+                    "str, list[str], list[str]",
+        default=dedent("""{
+            "repo_id": "Salesforce/xlam-function-calling-60k",
+            "config_name": "",
+            "commit_hash": "",
+            "attributes": {
+                "query_attr": "query",
+                "answers_attr": "answers",
+                "tools_attr": "tools"
+            }
+        }""")
+    )
+    augmentation_rate = DagParam(
+        description="(float) proportion of records to be augmented " + \
+                    "(x% of original dataset is created" + \
+                    " as additional augmented datapoints), i.e. " + \
+                    "truncated queries to serve as negative examples, " + \
+                    "meaning they trigger no tool call " + \
+                    "due to info incompleteness.",
+        default=.05
+    )
+    hf_enrich_dataset = DagParam(
+        description="dict with 'repo_id', 'config_name' and 'commit_hash', " + \
+                    "query_attribute' and 'query_attribute_handler' keys. " + \
+                    "if 'commit_hash is None, falls back to latest version " + \
+                    "of the dataset available in parquet format." + \
+                    "'query_attribute' depicts the dataset attribute " + \
+                    "from which 'queries' are to be sampled." + \
+                    "'query_attribute_handler' serves for attributes " + \
+                    "that have complex structure, " + \
+                    "other than 'string' datatype.",
+        # @see https://huggingface.co/datasets/google-research-datasets/natural_questions
+        default=dedent("""{
+            "repo_id": "lighteval/natural_questions_clean",
+            "config_name": "",
+            "commit_hash": "",
+            "query_attribute": "question",
+            "query_attribute_handler": "lambda x: x"
+        }""")
+    )
+    enrichment_rate = DagParam(
+        description="(float) proportion of records " + \
+                    "to be added from the 'hf_enrich_dataset'" + \
+                    "(x% of original dataset is sampled and" + \
+                    " added as enriching datapoints), i.e. " + \
+                    "queries to serve as negative examples, " + \
+                    "due to their complete disconnexion " + \
+                    "to tool calling situations.",
+        default=.1
+    )
+    dataset_repo_id = DagParam(
+        description="(str) The 'repo_id' to be used " + \
+                    "for the Hugging Face dataset version push " + \
+                    "(will be created at runtime" + \
+                    " if doesn't already exist).",
+        default="retrain-pipelines/func_calls"
+    )
+    polars_engine = DagParam(
+        description="The engine used by Polars for " + \
+                    "dataset querying and processing " + \
+                    "(either 'gpu' or 'cpu').",
+        default="gpu"
+    )
+    hf_base_model = DagParam(
+        description="(str) dict with 'repo_id' and 'commit_hash' keys." + \
+                    "if 'commit_hash is None, falls back " + \
+                    "to latest available version of the model.",
+        default=dedent("""{
+            "repo_id": "unsloth/Qwen2.5-1.5B",
+            "commit_hash": ""
+        }""")
+    )
+    cpt_training_args = DagParam(
+        description="dict with `TrainingArguments` params " + \
+                    "for the CPT job.",
+        default=dedent("""{
+            "warmup_ratio": 0.1,
+            "num_train_epochs": 1
+        }""")
+    )
+    sft_training_args = DagParam(
+        description="dict with `TrainingArguments` params " + \
+                    "for the SFT job.",
+        default=dedent("""{
+            "warmup_ratio": 0.1,
+            "num_train_epochs": 1
+        }""")
+    )
+    model_repo_id = DagParam(
+        description="(str) The 'repo_id' to be used " + \
+                    "for the Hugging Face model version push " + \
+                    "(will be created at runtime" + \
+                    " if doesn't already exist).",
+        default="retrain-pipelines/function_caller"
+    )
+    default_pipeline_card_module_dir = \
+        os.path.dirname(
+            importlib.util.find_spec(
+                f"retrain_pipelines.pipeline_card."+
+                f"{RETRAIN_PIPELINE_TYPE}"
+            ).origin)
+    pipeline_card_artifacts_path = DagParam(
+        description="pipeline_card artifacts location " + \
+                    "(i.e. dir hosting your optional " + \
+                    " custom documentation files :" + \
+                    " 'pipeline_card.py' and/or 'template.html'" + \
+                    " and/or 'model_readme.py'"+\
+                    " and/or 'model_readme_template.md'," + \
+                    " and/or 'dataset_readme.py'" + \
+                    " and/or 'dataset_readme_template.md' file), " + \
+                    "if different from default.",
+        default=default_pipeline_card_module_dir
+    )
+    # TODO  -  convert from class method to TBD
+    # @staticmethod
+    # def copy_default_dataset_readme_module(
+        # target_dir: str,
+        # exists_ok: bool = False
+    # ) -> None:
+        # os.makedirs(target_dir, exist_ok=True)
+        # if (
+            # not exists_ok and
+            # os.path.exists(os.path.join(target_dir, "dataset_readme.py"))
+        # ):
+            # print("File already exists. Skipping copy.")
+        # else:
+            # filefullname = os.path.join(
+                    # default_pipeline_card_module_dir,
+                    # "dataset_readme.py"
+                # )
+            # shutil.copy(filefullname, target_dir)
+            # print(filefullname)
+    # TODO  -  convert from class method to TBD
+    # @staticmethod
+    # def copy_default_dataset_readme_template(
+        # target_dir: str,
+        # exists_ok: bool = False
+    # ) -> None:
+        # os.makedirs(target_dir, exist_ok=True)
+        # if (
+            # not exists_ok and
+            # os.path.exists(os.path.join(target_dir,
+                                        # "dataset_readme_template.md"))
+        # ):
+            # print("File already exists. Skipping copy.")
+        # else:
+            # filefullname = os.path.join(
+                    # default_pipeline_card_module_dir,
+                    # "dataset_readme_template.md")
+            # shutil.copy(filefullname, target_dir)
+            # print(filefullname)
+    # TODO  -  convert from class method to TBD
+    # @staticmethod
+    # def copy_default_model_readme_module(
+        # target_dir: str,
+        # exists_ok: bool = False
+    # ) -> None:
+        # os.makedirs(target_dir, exist_ok=True)
+        # if (
+            # not exists_ok and
+            # os.path.exists(os.path.join(target_dir, "model_readme.py"))
+        # ):
+            # print("File already exists. Skipping copy.")
+        # else:
+            # filefullname = os.path.join(
+                    # default_pipeline_card_module_dir,
+                    # "model_readme.py"
+                # )
+            # shutil.copy(filefullname, target_dir)
+            # print(filefullname)
+    # TODO  -  convert from class method to TBD
+    # @staticmethod
+    # def copy_default_model_readme_template(
+        # target_dir: str,
+        # exists_ok: bool = False
+    # ) -> None:
+        # os.makedirs(target_dir, exist_ok=True)
+        # if (
+            # not exists_ok and
+            # os.path.exists(os.path.join(target_dir,
+                                        # "model_readme_template.md"))
+        # ):
+            # print("File already exists. Skipping copy.")
+        # else:
+            # filefullname = os.path.join(
+                    # default_pipeline_card_module_dir,
+                    # "model_readme_template.md")
+            # shutil.copy(filefullname, target_dir)
+            # print(filefullname)
+    # TODO  -  convert from class method to TBD
+    # @staticmethod
+    # def copy_default_pipeline_card_module(
+        # target_dir: str,
+        # exists_ok: bool = False
+    # ) -> None:
+        # os.makedirs(target_dir, exist_ok=True)
+        # if (
+            # not exists_ok and
+            # os.path.exists(os.path.join(target_dir, "pipeline_card.py"))
+        # ):
+            # print("File already exists. Skipping copy.")
+        # else:
+            # filefullname = os.path.join(
+                    # default_pipeline_card_module_dir,
+                    # "pipeline_card.py"
+                # )
+            # shutil.copy(filefullname, target_dir)
+            # print(filefullname)
+    # TODO  -  convert from class method to TBD
+    # @staticmethod
+    # def copy_default_pipeline_card_html_template(
+        # target_dir: str,
+        # exists_ok: bool = False
+    # ) -> None:
+        # os.makedirs(target_dir, exist_ok=True)
+        # if (
+            # not exists_ok and
+            # os.path.exists(os.path.join(target_dir, "template.html"))
+        # ):
+            # print("File already exists. Skipping copy.")
+        # else:
+            # filefullname = os.path.join(
+                    # default_pipeline_card_module_dir,
+                    # "template.html")
+            # shutil.copy(filefullname, target_dir)
+            # print(filefullname)
+    del RETRAIN_PIPELINE_TYPE
+    #---------------------------------------------------------------------------
+    return start >> eda \
+            >> augment_data >> enrich_data >> dataset_to_hub \
+            >> continued_pre_training >> supervised_finetuning \
+            >> evaluate_model >> model_version_blessing \
+            >> model_to_hub >> infra_validator >> pipeline_card \
+            >> pipeline_to_hub >> deploy >> load_test >> end