Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5690252b01a9287a242580edaed9ff30af1008823786e04a65b411b59ecfdc44
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:457d3b8c3175f53ad489b3520fc0eb17b28944f92f6f16f021081395c6aa81fc
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41246fd78fb0ee30b2b8b77e7a74c960a74ee8d37fb24fc3ce85a34f6c70f6ad
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2f1a3a6b40a0cfc5303e21d56df8a5f6a7be871ab1a7c7ef5f6e19db3fcc37d
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6edafd2cebddb701439455b36f588ba3cec151ef0275a5fc28b2555905a52133
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/trainer_state.json

@@ -0,0 +1,831 @@
+{
+  "best_metric": 1.0289618968963623,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303",
+  "epoch": 6.996699669966997,
+  "eval_steps": 10,
+  "global_step": 1060,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.066006600660066,
+      "grad_norm": 0.8332476615905762,
+      "learning_rate": 0.0002,
+      "loss": 1.6257,
+      "step": 10
+    },
+    {
+      "epoch": 0.132013201320132,
+      "grad_norm": 1.1134452819824219,
+      "learning_rate": 0.0002,
+      "loss": 1.3155,
+      "step": 20
+    },
+    {
+      "epoch": 0.19801980198019803,
+      "grad_norm": 0.8637537956237793,
+      "learning_rate": 0.0002,
+      "loss": 1.2979,
+      "step": 30
+    },
+    {
+      "epoch": 0.264026402640264,
+      "grad_norm": 1.291804313659668,
+      "learning_rate": 0.0002,
+      "loss": 1.1495,
+      "step": 40
+    },
+    {
+      "epoch": 0.33003300330033003,
+      "grad_norm": 0.6008025407791138,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 50
+    },
+    {
+      "epoch": 0.39603960396039606,
+      "grad_norm": 0.6817359924316406,
+      "learning_rate": 0.0002,
+      "loss": 1.0974,
+      "step": 60
+    },
+    {
+      "epoch": 0.46204620462046203,
+      "grad_norm": 0.7527281641960144,
+      "learning_rate": 0.0002,
+      "loss": 1.0563,
+      "step": 70
+    },
+    {
+      "epoch": 0.528052805280528,
+      "grad_norm": 0.6140271425247192,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 80
+    },
+    {
+      "epoch": 0.594059405940594,
+      "grad_norm": 0.6146464943885803,
+      "learning_rate": 0.0002,
+      "loss": 1.0977,
+      "step": 90
+    },
+    {
+      "epoch": 0.6600660066006601,
+      "grad_norm": 0.6727924942970276,
+      "learning_rate": 0.0002,
+      "loss": 1.1484,
+      "step": 100
+    },
+    {
+      "epoch": 0.7260726072607261,
+      "grad_norm": 0.6649640798568726,
+      "learning_rate": 0.0002,
+      "loss": 1.0635,
+      "step": 110
+    },
+    {
+      "epoch": 0.7920792079207921,
+      "grad_norm": 0.5999083518981934,
+      "learning_rate": 0.0002,
+      "loss": 1.0545,
+      "step": 120
+    },
+    {
+      "epoch": 0.858085808580858,
+      "grad_norm": 0.638982355594635,
+      "learning_rate": 0.0002,
+      "loss": 1.0681,
+      "step": 130
+    },
+    {
+      "epoch": 0.9240924092409241,
+      "grad_norm": 0.604159951210022,
+      "learning_rate": 0.0002,
+      "loss": 1.0369,
+      "step": 140
+    },
+    {
+      "epoch": 0.9900990099009901,
+      "grad_norm": 0.7679715752601624,
+      "learning_rate": 0.0002,
+      "loss": 1.0963,
+      "step": 150
+    },
+    {
+      "epoch": 0.9966996699669967,
+      "eval_loss": 1.0424563884735107,
+      "eval_runtime": 7.9324,
+      "eval_samples_per_second": 13.741,
+      "eval_steps_per_second": 1.765,
+      "step": 151
+    },
+    {
+      "epoch": 1.056105610561056,
+      "grad_norm": 0.9450559616088867,
+      "learning_rate": 0.0002,
+      "loss": 0.9183,
+      "step": 160
+    },
+    {
+      "epoch": 1.1221122112211221,
+      "grad_norm": 0.6976049542427063,
+      "learning_rate": 0.0002,
+      "loss": 0.9742,
+      "step": 170
+    },
+    {
+      "epoch": 1.188118811881188,
+      "grad_norm": 0.6649869680404663,
+      "learning_rate": 0.0002,
+      "loss": 0.9278,
+      "step": 180
+    },
+    {
+      "epoch": 1.2541254125412542,
+      "grad_norm": 0.7072765827178955,
+      "learning_rate": 0.0002,
+      "loss": 0.961,
+      "step": 190
+    },
+    {
+      "epoch": 1.3201320132013201,
+      "grad_norm": 0.7633873224258423,
+      "learning_rate": 0.0002,
+      "loss": 0.9627,
+      "step": 200
+    },
+    {
+      "epoch": 1.386138613861386,
+      "grad_norm": 0.7836750149726868,
+      "learning_rate": 0.0002,
+      "loss": 0.9528,
+      "step": 210
+    },
+    {
+      "epoch": 1.4521452145214522,
+      "grad_norm": 0.6136282682418823,
+      "learning_rate": 0.0002,
+      "loss": 0.9099,
+      "step": 220
+    },
+    {
+      "epoch": 1.5181518151815183,
+      "grad_norm": 0.6747711300849915,
+      "learning_rate": 0.0002,
+      "loss": 0.9719,
+      "step": 230
+    },
+    {
+      "epoch": 1.5841584158415842,
+      "grad_norm": 0.8176482319831848,
+      "learning_rate": 0.0002,
+      "loss": 0.953,
+      "step": 240
+    },
+    {
+      "epoch": 1.6501650165016502,
+      "grad_norm": 0.5480154156684875,
+      "learning_rate": 0.0002,
+      "loss": 0.9546,
+      "step": 250
+    },
+    {
+      "epoch": 1.716171617161716,
+      "grad_norm": 0.6764057874679565,
+      "learning_rate": 0.0002,
+      "loss": 0.932,
+      "step": 260
+    },
+    {
+      "epoch": 1.7821782178217822,
+      "grad_norm": 0.8389566540718079,
+      "learning_rate": 0.0002,
+      "loss": 1.0034,
+      "step": 270
+    },
+    {
+      "epoch": 1.8481848184818483,
+      "grad_norm": 0.6570506691932678,
+      "learning_rate": 0.0002,
+      "loss": 0.9758,
+      "step": 280
+    },
+    {
+      "epoch": 1.9141914191419143,
+      "grad_norm": 0.7700225710868835,
+      "learning_rate": 0.0002,
+      "loss": 1.023,
+      "step": 290
+    },
+    {
+      "epoch": 1.9801980198019802,
+      "grad_norm": 0.6293354630470276,
+      "learning_rate": 0.0002,
+      "loss": 0.971,
+      "step": 300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0289618968963623,
+      "eval_runtime": 7.9336,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 303
+    },
+    {
+      "epoch": 2.046204620462046,
+      "grad_norm": 0.69588303565979,
+      "learning_rate": 0.0002,
+      "loss": 0.9327,
+      "step": 310
+    },
+    {
+      "epoch": 2.112211221122112,
+      "grad_norm": 0.7868685126304626,
+      "learning_rate": 0.0002,
+      "loss": 0.811,
+      "step": 320
+    },
+    {
+      "epoch": 2.1782178217821784,
+      "grad_norm": 0.9364991188049316,
+      "learning_rate": 0.0002,
+      "loss": 0.7774,
+      "step": 330
+    },
+    {
+      "epoch": 2.2442244224422443,
+      "grad_norm": 0.7622379064559937,
+      "learning_rate": 0.0002,
+      "loss": 0.8324,
+      "step": 340
+    },
+    {
+      "epoch": 2.31023102310231,
+      "grad_norm": 0.7931517362594604,
+      "learning_rate": 0.0002,
+      "loss": 0.8191,
+      "step": 350
+    },
+    {
+      "epoch": 2.376237623762376,
+      "grad_norm": 1.2253040075302124,
+      "learning_rate": 0.0002,
+      "loss": 0.8314,
+      "step": 360
+    },
+    {
+      "epoch": 2.442244224422442,
+      "grad_norm": 0.9045709371566772,
+      "learning_rate": 0.0002,
+      "loss": 0.8224,
+      "step": 370
+    },
+    {
+      "epoch": 2.5082508250825084,
+      "grad_norm": 0.7717004418373108,
+      "learning_rate": 0.0002,
+      "loss": 0.7362,
+      "step": 380
+    },
+    {
+      "epoch": 2.5742574257425743,
+      "grad_norm": 0.9297442436218262,
+      "learning_rate": 0.0002,
+      "loss": 0.8469,
+      "step": 390
+    },
+    {
+      "epoch": 2.6402640264026402,
+      "grad_norm": 1.0603872537612915,
+      "learning_rate": 0.0002,
+      "loss": 0.7798,
+      "step": 400
+    },
+    {
+      "epoch": 2.706270627062706,
+      "grad_norm": 0.9738940000534058,
+      "learning_rate": 0.0002,
+      "loss": 0.8059,
+      "step": 410
+    },
+    {
+      "epoch": 2.772277227722772,
+      "grad_norm": 0.75095534324646,
+      "learning_rate": 0.0002,
+      "loss": 0.8192,
+      "step": 420
+    },
+    {
+      "epoch": 2.8382838283828384,
+      "grad_norm": 1.00453782081604,
+      "learning_rate": 0.0002,
+      "loss": 0.7972,
+      "step": 430
+    },
+    {
+      "epoch": 2.9042904290429044,
+      "grad_norm": 0.8055713772773743,
+      "learning_rate": 0.0002,
+      "loss": 0.8355,
+      "step": 440
+    },
+    {
+      "epoch": 2.9702970297029703,
+      "grad_norm": 0.7566568851470947,
+      "learning_rate": 0.0002,
+      "loss": 0.8208,
+      "step": 450
+    },
+    {
+      "epoch": 2.9966996699669965,
+      "eval_loss": 1.059197187423706,
+      "eval_runtime": 7.9351,
+      "eval_samples_per_second": 13.736,
+      "eval_steps_per_second": 1.764,
+      "step": 454
+    },
+    {
+      "epoch": 3.036303630363036,
+      "grad_norm": 1.0427310466766357,
+      "learning_rate": 0.0002,
+      "loss": 0.7221,
+      "step": 460
+    },
+    {
+      "epoch": 3.102310231023102,
+      "grad_norm": 0.9990588426589966,
+      "learning_rate": 0.0002,
+      "loss": 0.7019,
+      "step": 470
+    },
+    {
+      "epoch": 3.1683168316831685,
+      "grad_norm": 1.2692598104476929,
+      "learning_rate": 0.0002,
+      "loss": 0.675,
+      "step": 480
+    },
+    {
+      "epoch": 3.2343234323432344,
+      "grad_norm": 1.0239098072052002,
+      "learning_rate": 0.0002,
+      "loss": 0.6766,
+      "step": 490
+    },
+    {
+      "epoch": 3.3003300330033003,
+      "grad_norm": 0.9614437222480774,
+      "learning_rate": 0.0002,
+      "loss": 0.6614,
+      "step": 500
+    },
+    {
+      "epoch": 3.366336633663366,
+      "grad_norm": 1.0569125413894653,
+      "learning_rate": 0.0002,
+      "loss": 0.6609,
+      "step": 510
+    },
+    {
+      "epoch": 3.432343234323432,
+      "grad_norm": 0.9972309470176697,
+      "learning_rate": 0.0002,
+      "loss": 0.6279,
+      "step": 520
+    },
+    {
+      "epoch": 3.4983498349834985,
+      "grad_norm": 1.2322218418121338,
+      "learning_rate": 0.0002,
+      "loss": 0.6348,
+      "step": 530
+    },
+    {
+      "epoch": 3.5643564356435644,
+      "grad_norm": 1.422387957572937,
+      "learning_rate": 0.0002,
+      "loss": 0.6453,
+      "step": 540
+    },
+    {
+      "epoch": 3.6303630363036303,
+      "grad_norm": 1.4984462261199951,
+      "learning_rate": 0.0002,
+      "loss": 0.6521,
+      "step": 550
+    },
+    {
+      "epoch": 3.6963696369636962,
+      "grad_norm": 1.0426137447357178,
+      "learning_rate": 0.0002,
+      "loss": 0.6864,
+      "step": 560
+    },
+    {
+      "epoch": 3.762376237623762,
+      "grad_norm": 1.1011866331100464,
+      "learning_rate": 0.0002,
+      "loss": 0.6486,
+      "step": 570
+    },
+    {
+      "epoch": 3.8283828382838285,
+      "grad_norm": 1.016489028930664,
+      "learning_rate": 0.0002,
+      "loss": 0.673,
+      "step": 580
+    },
+    {
+      "epoch": 3.8943894389438944,
+      "grad_norm": 1.064589262008667,
+      "learning_rate": 0.0002,
+      "loss": 0.6675,
+      "step": 590
+    },
+    {
+      "epoch": 3.9603960396039604,
+      "grad_norm": 1.0282487869262695,
+      "learning_rate": 0.0002,
+      "loss": 0.7032,
+      "step": 600
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.1354578733444214,
+      "eval_runtime": 7.9345,
+      "eval_samples_per_second": 13.737,
+      "eval_steps_per_second": 1.764,
+      "step": 606
+    },
+    {
+      "epoch": 4.026402640264027,
+      "grad_norm": 1.0715293884277344,
+      "learning_rate": 0.0002,
+      "loss": 0.6208,
+      "step": 610
+    },
+    {
+      "epoch": 4.092409240924092,
+      "grad_norm": 1.0794376134872437,
+      "learning_rate": 0.0002,
+      "loss": 0.5347,
+      "step": 620
+    },
+    {
+      "epoch": 4.158415841584159,
+      "grad_norm": 1.2978978157043457,
+      "learning_rate": 0.0002,
+      "loss": 0.5038,
+      "step": 630
+    },
+    {
+      "epoch": 4.224422442244224,
+      "grad_norm": 1.1555476188659668,
+      "learning_rate": 0.0002,
+      "loss": 0.5086,
+      "step": 640
+    },
+    {
+      "epoch": 4.29042904290429,
+      "grad_norm": 1.2701029777526855,
+      "learning_rate": 0.0002,
+      "loss": 0.515,
+      "step": 650
+    },
+    {
+      "epoch": 4.356435643564357,
+      "grad_norm": 1.2661395072937012,
+      "learning_rate": 0.0002,
+      "loss": 0.5166,
+      "step": 660
+    },
+    {
+      "epoch": 4.422442244224422,
+      "grad_norm": 1.5223569869995117,
+      "learning_rate": 0.0002,
+      "loss": 0.5345,
+      "step": 670
+    },
+    {
+      "epoch": 4.488448844884489,
+      "grad_norm": 1.1259031295776367,
+      "learning_rate": 0.0002,
+      "loss": 0.5093,
+      "step": 680
+    },
+    {
+      "epoch": 4.554455445544555,
+      "grad_norm": 1.1892699003219604,
+      "learning_rate": 0.0002,
+      "loss": 0.5252,
+      "step": 690
+    },
+    {
+      "epoch": 4.62046204620462,
+      "grad_norm": 1.031516671180725,
+      "learning_rate": 0.0002,
+      "loss": 0.5374,
+      "step": 700
+    },
+    {
+      "epoch": 4.686468646864687,
+      "grad_norm": 1.576996922492981,
+      "learning_rate": 0.0002,
+      "loss": 0.5511,
+      "step": 710
+    },
+    {
+      "epoch": 4.752475247524752,
+      "grad_norm": 1.208756923675537,
+      "learning_rate": 0.0002,
+      "loss": 0.5371,
+      "step": 720
+    },
+    {
+      "epoch": 4.818481848184819,
+      "grad_norm": 1.4121363162994385,
+      "learning_rate": 0.0002,
+      "loss": 0.543,
+      "step": 730
+    },
+    {
+      "epoch": 4.884488448844884,
+      "grad_norm": 1.2884345054626465,
+      "learning_rate": 0.0002,
+      "loss": 0.5483,
+      "step": 740
+    },
+    {
+      "epoch": 4.9504950495049505,
+      "grad_norm": 1.3778842687606812,
+      "learning_rate": 0.0002,
+      "loss": 0.5287,
+      "step": 750
+    },
+    {
+      "epoch": 4.996699669966997,
+      "eval_loss": 1.2193516492843628,
+      "eval_runtime": 7.9403,
+      "eval_samples_per_second": 13.727,
+      "eval_steps_per_second": 1.763,
+      "step": 757
+    },
+    {
+      "epoch": 5.016501650165017,
+      "grad_norm": 1.1151881217956543,
+      "learning_rate": 0.0002,
+      "loss": 0.525,
+      "step": 760
+    },
+    {
+      "epoch": 5.082508250825082,
+      "grad_norm": 1.3323009014129639,
+      "learning_rate": 0.0002,
+      "loss": 0.4187,
+      "step": 770
+    },
+    {
+      "epoch": 5.148514851485149,
+      "grad_norm": 1.7906690835952759,
+      "learning_rate": 0.0002,
+      "loss": 0.4083,
+      "step": 780
+    },
+    {
+      "epoch": 5.214521452145214,
+      "grad_norm": 1.215036153793335,
+      "learning_rate": 0.0002,
+      "loss": 0.3952,
+      "step": 790
+    },
+    {
+      "epoch": 5.2805280528052805,
+      "grad_norm": 1.3145633935928345,
+      "learning_rate": 0.0002,
+      "loss": 0.4021,
+      "step": 800
+    },
+    {
+      "epoch": 5.346534653465347,
+      "grad_norm": 1.2486501932144165,
+      "learning_rate": 0.0002,
+      "loss": 0.4073,
+      "step": 810
+    },
+    {
+      "epoch": 5.412541254125412,
+      "grad_norm": 1.5110037326812744,
+      "learning_rate": 0.0002,
+      "loss": 0.4297,
+      "step": 820
+    },
+    {
+      "epoch": 5.478547854785479,
+      "grad_norm": 1.030617594718933,
+      "learning_rate": 0.0002,
+      "loss": 0.418,
+      "step": 830
+    },
+    {
+      "epoch": 5.544554455445544,
+      "grad_norm": 1.6717298030853271,
+      "learning_rate": 0.0002,
+      "loss": 0.4247,
+      "step": 840
+    },
+    {
+      "epoch": 5.6105610561056105,
+      "grad_norm": 1.4030007123947144,
+      "learning_rate": 0.0002,
+      "loss": 0.4286,
+      "step": 850
+    },
+    {
+      "epoch": 5.676567656765677,
+      "grad_norm": 1.288204550743103,
+      "learning_rate": 0.0002,
+      "loss": 0.4262,
+      "step": 860
+    },
+    {
+      "epoch": 5.742574257425742,
+      "grad_norm": 1.0709242820739746,
+      "learning_rate": 0.0002,
+      "loss": 0.4389,
+      "step": 870
+    },
+    {
+      "epoch": 5.808580858085809,
+      "grad_norm": 1.2238759994506836,
+      "learning_rate": 0.0002,
+      "loss": 0.434,
+      "step": 880
+    },
+    {
+      "epoch": 5.874587458745875,
+      "grad_norm": 1.4378324747085571,
+      "learning_rate": 0.0002,
+      "loss": 0.436,
+      "step": 890
+    },
+    {
+      "epoch": 5.9405940594059405,
+      "grad_norm": 1.6783941984176636,
+      "learning_rate": 0.0002,
+      "loss": 0.4282,
+      "step": 900
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.3334790468215942,
+      "eval_runtime": 7.9337,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 909
+    },
+    {
+      "epoch": 6.006600660066007,
+      "grad_norm": 1.2153123617172241,
+      "learning_rate": 0.0002,
+      "loss": 0.4035,
+      "step": 910
+    },
+    {
+      "epoch": 6.072607260726072,
+      "grad_norm": 1.8521552085876465,
+      "learning_rate": 0.0002,
+      "loss": 0.3311,
+      "step": 920
+    },
+    {
+      "epoch": 6.138613861386139,
+      "grad_norm": 1.5133066177368164,
+      "learning_rate": 0.0002,
+      "loss": 0.3199,
+      "step": 930
+    },
+    {
+      "epoch": 6.204620462046204,
+      "grad_norm": 1.724316120147705,
+      "learning_rate": 0.0002,
+      "loss": 0.3108,
+      "step": 940
+    },
+    {
+      "epoch": 6.270627062706271,
+      "grad_norm": 1.2472296953201294,
+      "learning_rate": 0.0002,
+      "loss": 0.3375,
+      "step": 950
+    },
+    {
+      "epoch": 6.336633663366337,
+      "grad_norm": 1.9551066160202026,
+      "learning_rate": 0.0002,
+      "loss": 0.3529,
+      "step": 960
+    },
+    {
+      "epoch": 6.402640264026402,
+      "grad_norm": 1.3545963764190674,
+      "learning_rate": 0.0002,
+      "loss": 0.35,
+      "step": 970
+    },
+    {
+      "epoch": 6.468646864686469,
+      "grad_norm": 1.4807971715927124,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 980
+    },
+    {
+      "epoch": 6.534653465346535,
+      "grad_norm": 1.6042426824569702,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 990
+    },
+    {
+      "epoch": 6.600660066006601,
+      "grad_norm": 1.5707716941833496,
+      "learning_rate": 0.0002,
+      "loss": 0.3632,
+      "step": 1000
+    },
+    {
+      "epoch": 6.666666666666667,
+      "grad_norm": 1.5197912454605103,
+      "learning_rate": 0.0002,
+      "loss": 0.3663,
+      "step": 1010
+    },
+    {
+      "epoch": 6.732673267326732,
+      "grad_norm": 1.9985229969024658,
+      "learning_rate": 0.0002,
+      "loss": 0.3497,
+      "step": 1020
+    },
+    {
+      "epoch": 6.798679867986799,
+      "grad_norm": 1.6729710102081299,
+      "learning_rate": 0.0002,
+      "loss": 0.346,
+      "step": 1030
+    },
+    {
+      "epoch": 6.864686468646864,
+      "grad_norm": 1.3435407876968384,
+      "learning_rate": 0.0002,
+      "loss": 0.3737,
+      "step": 1040
+    },
+    {
+      "epoch": 6.930693069306931,
+      "grad_norm": 1.4809728860855103,
+      "learning_rate": 0.0002,
+      "loss": 0.3646,
+      "step": 1050
+    },
+    {
+      "epoch": 6.996699669966997,
+      "grad_norm": 1.7941899299621582,
+      "learning_rate": 0.0002,
+      "loss": 0.3519,
+      "step": 1060
+    },
+    {
+      "epoch": 6.996699669966997,
+      "eval_loss": 1.4348182678222656,
+      "eval_runtime": 7.9375,
+      "eval_samples_per_second": 13.732,
+      "eval_steps_per_second": 1.764,
+      "step": 1060
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.65291476729856e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1060/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2de0dfe312263bfc3e8d6e80fdd97c7a6f0bdac64594686d024b1965cb27e1a6
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cf186c38bcd276a7ba7b3abed0cbe7141f5167cc27565c6ac9be3bb34105887
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24ca5b9d5e8886d03316a56a9e734dd2c30a3b8fd272428ea90339164e9538cf
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f081b4b825f2aac8d55b5d9240b6b55e7a3ee91638ade10d77f0505833b35af
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9e9e2f6703e5f14b0b4760300d55023da415b6fdb536d70f6ee7afecfbf169a
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/trainer_state.json

@@ -0,0 +1,937 @@
+{
+  "best_metric": 1.0289618968963623,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303",
+  "epoch": 7.973597359735973,
+  "eval_steps": 10,
+  "global_step": 1208,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.066006600660066,
+      "grad_norm": 0.8332476615905762,
+      "learning_rate": 0.0002,
+      "loss": 1.6257,
+      "step": 10
+    },
+    {
+      "epoch": 0.132013201320132,
+      "grad_norm": 1.1134452819824219,
+      "learning_rate": 0.0002,
+      "loss": 1.3155,
+      "step": 20
+    },
+    {
+      "epoch": 0.19801980198019803,
+      "grad_norm": 0.8637537956237793,
+      "learning_rate": 0.0002,
+      "loss": 1.2979,
+      "step": 30
+    },
+    {
+      "epoch": 0.264026402640264,
+      "grad_norm": 1.291804313659668,
+      "learning_rate": 0.0002,
+      "loss": 1.1495,
+      "step": 40
+    },
+    {
+      "epoch": 0.33003300330033003,
+      "grad_norm": 0.6008025407791138,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 50
+    },
+    {
+      "epoch": 0.39603960396039606,
+      "grad_norm": 0.6817359924316406,
+      "learning_rate": 0.0002,
+      "loss": 1.0974,
+      "step": 60
+    },
+    {
+      "epoch": 0.46204620462046203,
+      "grad_norm": 0.7527281641960144,
+      "learning_rate": 0.0002,
+      "loss": 1.0563,
+      "step": 70
+    },
+    {
+      "epoch": 0.528052805280528,
+      "grad_norm": 0.6140271425247192,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 80
+    },
+    {
+      "epoch": 0.594059405940594,
+      "grad_norm": 0.6146464943885803,
+      "learning_rate": 0.0002,
+      "loss": 1.0977,
+      "step": 90
+    },
+    {
+      "epoch": 0.6600660066006601,
+      "grad_norm": 0.6727924942970276,
+      "learning_rate": 0.0002,
+      "loss": 1.1484,
+      "step": 100
+    },
+    {
+      "epoch": 0.7260726072607261,
+      "grad_norm": 0.6649640798568726,
+      "learning_rate": 0.0002,
+      "loss": 1.0635,
+      "step": 110
+    },
+    {
+      "epoch": 0.7920792079207921,
+      "grad_norm": 0.5999083518981934,
+      "learning_rate": 0.0002,
+      "loss": 1.0545,
+      "step": 120
+    },
+    {
+      "epoch": 0.858085808580858,
+      "grad_norm": 0.638982355594635,
+      "learning_rate": 0.0002,
+      "loss": 1.0681,
+      "step": 130
+    },
+    {
+      "epoch": 0.9240924092409241,
+      "grad_norm": 0.604159951210022,
+      "learning_rate": 0.0002,
+      "loss": 1.0369,
+      "step": 140
+    },
+    {
+      "epoch": 0.9900990099009901,
+      "grad_norm": 0.7679715752601624,
+      "learning_rate": 0.0002,
+      "loss": 1.0963,
+      "step": 150
+    },
+    {
+      "epoch": 0.9966996699669967,
+      "eval_loss": 1.0424563884735107,
+      "eval_runtime": 7.9324,
+      "eval_samples_per_second": 13.741,
+      "eval_steps_per_second": 1.765,
+      "step": 151
+    },
+    {
+      "epoch": 1.056105610561056,
+      "grad_norm": 0.9450559616088867,
+      "learning_rate": 0.0002,
+      "loss": 0.9183,
+      "step": 160
+    },
+    {
+      "epoch": 1.1221122112211221,
+      "grad_norm": 0.6976049542427063,
+      "learning_rate": 0.0002,
+      "loss": 0.9742,
+      "step": 170
+    },
+    {
+      "epoch": 1.188118811881188,
+      "grad_norm": 0.6649869680404663,
+      "learning_rate": 0.0002,
+      "loss": 0.9278,
+      "step": 180
+    },
+    {
+      "epoch": 1.2541254125412542,
+      "grad_norm": 0.7072765827178955,
+      "learning_rate": 0.0002,
+      "loss": 0.961,
+      "step": 190
+    },
+    {
+      "epoch": 1.3201320132013201,
+      "grad_norm": 0.7633873224258423,
+      "learning_rate": 0.0002,
+      "loss": 0.9627,
+      "step": 200
+    },
+    {
+      "epoch": 1.386138613861386,
+      "grad_norm": 0.7836750149726868,
+      "learning_rate": 0.0002,
+      "loss": 0.9528,
+      "step": 210
+    },
+    {
+      "epoch": 1.4521452145214522,
+      "grad_norm": 0.6136282682418823,
+      "learning_rate": 0.0002,
+      "loss": 0.9099,
+      "step": 220
+    },
+    {
+      "epoch": 1.5181518151815183,
+      "grad_norm": 0.6747711300849915,
+      "learning_rate": 0.0002,
+      "loss": 0.9719,
+      "step": 230
+    },
+    {
+      "epoch": 1.5841584158415842,
+      "grad_norm": 0.8176482319831848,
+      "learning_rate": 0.0002,
+      "loss": 0.953,
+      "step": 240
+    },
+    {
+      "epoch": 1.6501650165016502,
+      "grad_norm": 0.5480154156684875,
+      "learning_rate": 0.0002,
+      "loss": 0.9546,
+      "step": 250
+    },
+    {
+      "epoch": 1.716171617161716,
+      "grad_norm": 0.6764057874679565,
+      "learning_rate": 0.0002,
+      "loss": 0.932,
+      "step": 260
+    },
+    {
+      "epoch": 1.7821782178217822,
+      "grad_norm": 0.8389566540718079,
+      "learning_rate": 0.0002,
+      "loss": 1.0034,
+      "step": 270
+    },
+    {
+      "epoch": 1.8481848184818483,
+      "grad_norm": 0.6570506691932678,
+      "learning_rate": 0.0002,
+      "loss": 0.9758,
+      "step": 280
+    },
+    {
+      "epoch": 1.9141914191419143,
+      "grad_norm": 0.7700225710868835,
+      "learning_rate": 0.0002,
+      "loss": 1.023,
+      "step": 290
+    },
+    {
+      "epoch": 1.9801980198019802,
+      "grad_norm": 0.6293354630470276,
+      "learning_rate": 0.0002,
+      "loss": 0.971,
+      "step": 300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0289618968963623,
+      "eval_runtime": 7.9336,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 303
+    },
+    {
+      "epoch": 2.046204620462046,
+      "grad_norm": 0.69588303565979,
+      "learning_rate": 0.0002,
+      "loss": 0.9327,
+      "step": 310
+    },
+    {
+      "epoch": 2.112211221122112,
+      "grad_norm": 0.7868685126304626,
+      "learning_rate": 0.0002,
+      "loss": 0.811,
+      "step": 320
+    },
+    {
+      "epoch": 2.1782178217821784,
+      "grad_norm": 0.9364991188049316,
+      "learning_rate": 0.0002,
+      "loss": 0.7774,
+      "step": 330
+    },
+    {
+      "epoch": 2.2442244224422443,
+      "grad_norm": 0.7622379064559937,
+      "learning_rate": 0.0002,
+      "loss": 0.8324,
+      "step": 340
+    },
+    {
+      "epoch": 2.31023102310231,
+      "grad_norm": 0.7931517362594604,
+      "learning_rate": 0.0002,
+      "loss": 0.8191,
+      "step": 350
+    },
+    {
+      "epoch": 2.376237623762376,
+      "grad_norm": 1.2253040075302124,
+      "learning_rate": 0.0002,
+      "loss": 0.8314,
+      "step": 360
+    },
+    {
+      "epoch": 2.442244224422442,
+      "grad_norm": 0.9045709371566772,
+      "learning_rate": 0.0002,
+      "loss": 0.8224,
+      "step": 370
+    },
+    {
+      "epoch": 2.5082508250825084,
+      "grad_norm": 0.7717004418373108,
+      "learning_rate": 0.0002,
+      "loss": 0.7362,
+      "step": 380
+    },
+    {
+      "epoch": 2.5742574257425743,
+      "grad_norm": 0.9297442436218262,
+      "learning_rate": 0.0002,
+      "loss": 0.8469,
+      "step": 390
+    },
+    {
+      "epoch": 2.6402640264026402,
+      "grad_norm": 1.0603872537612915,
+      "learning_rate": 0.0002,
+      "loss": 0.7798,
+      "step": 400
+    },
+    {
+      "epoch": 2.706270627062706,
+      "grad_norm": 0.9738940000534058,
+      "learning_rate": 0.0002,
+      "loss": 0.8059,
+      "step": 410
+    },
+    {
+      "epoch": 2.772277227722772,
+      "grad_norm": 0.75095534324646,
+      "learning_rate": 0.0002,
+      "loss": 0.8192,
+      "step": 420
+    },
+    {
+      "epoch": 2.8382838283828384,
+      "grad_norm": 1.00453782081604,
+      "learning_rate": 0.0002,
+      "loss": 0.7972,
+      "step": 430
+    },
+    {
+      "epoch": 2.9042904290429044,
+      "grad_norm": 0.8055713772773743,
+      "learning_rate": 0.0002,
+      "loss": 0.8355,
+      "step": 440
+    },
+    {
+      "epoch": 2.9702970297029703,
+      "grad_norm": 0.7566568851470947,
+      "learning_rate": 0.0002,
+      "loss": 0.8208,
+      "step": 450
+    },
+    {
+      "epoch": 2.9966996699669965,
+      "eval_loss": 1.059197187423706,
+      "eval_runtime": 7.9351,
+      "eval_samples_per_second": 13.736,
+      "eval_steps_per_second": 1.764,
+      "step": 454
+    },
+    {
+      "epoch": 3.036303630363036,
+      "grad_norm": 1.0427310466766357,
+      "learning_rate": 0.0002,
+      "loss": 0.7221,
+      "step": 460
+    },
+    {
+      "epoch": 3.102310231023102,
+      "grad_norm": 0.9990588426589966,
+      "learning_rate": 0.0002,
+      "loss": 0.7019,
+      "step": 470
+    },
+    {
+      "epoch": 3.1683168316831685,
+      "grad_norm": 1.2692598104476929,
+      "learning_rate": 0.0002,
+      "loss": 0.675,
+      "step": 480
+    },
+    {
+      "epoch": 3.2343234323432344,
+      "grad_norm": 1.0239098072052002,
+      "learning_rate": 0.0002,
+      "loss": 0.6766,
+      "step": 490
+    },
+    {
+      "epoch": 3.3003300330033003,
+      "grad_norm": 0.9614437222480774,
+      "learning_rate": 0.0002,
+      "loss": 0.6614,
+      "step": 500
+    },
+    {
+      "epoch": 3.366336633663366,
+      "grad_norm": 1.0569125413894653,
+      "learning_rate": 0.0002,
+      "loss": 0.6609,
+      "step": 510
+    },
+    {
+      "epoch": 3.432343234323432,
+      "grad_norm": 0.9972309470176697,
+      "learning_rate": 0.0002,
+      "loss": 0.6279,
+      "step": 520
+    },
+    {
+      "epoch": 3.4983498349834985,
+      "grad_norm": 1.2322218418121338,
+      "learning_rate": 0.0002,
+      "loss": 0.6348,
+      "step": 530
+    },
+    {
+      "epoch": 3.5643564356435644,
+      "grad_norm": 1.422387957572937,
+      "learning_rate": 0.0002,
+      "loss": 0.6453,
+      "step": 540
+    },
+    {
+      "epoch": 3.6303630363036303,
+      "grad_norm": 1.4984462261199951,
+      "learning_rate": 0.0002,
+      "loss": 0.6521,
+      "step": 550
+    },
+    {
+      "epoch": 3.6963696369636962,
+      "grad_norm": 1.0426137447357178,
+      "learning_rate": 0.0002,
+      "loss": 0.6864,
+      "step": 560
+    },
+    {
+      "epoch": 3.762376237623762,
+      "grad_norm": 1.1011866331100464,
+      "learning_rate": 0.0002,
+      "loss": 0.6486,
+      "step": 570
+    },
+    {
+      "epoch": 3.8283828382838285,
+      "grad_norm": 1.016489028930664,
+      "learning_rate": 0.0002,
+      "loss": 0.673,
+      "step": 580
+    },
+    {
+      "epoch": 3.8943894389438944,
+      "grad_norm": 1.064589262008667,
+      "learning_rate": 0.0002,
+      "loss": 0.6675,
+      "step": 590
+    },
+    {
+      "epoch": 3.9603960396039604,
+      "grad_norm": 1.0282487869262695,
+      "learning_rate": 0.0002,
+      "loss": 0.7032,
+      "step": 600
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.1354578733444214,
+      "eval_runtime": 7.9345,
+      "eval_samples_per_second": 13.737,
+      "eval_steps_per_second": 1.764,
+      "step": 606
+    },
+    {
+      "epoch": 4.026402640264027,
+      "grad_norm": 1.0715293884277344,
+      "learning_rate": 0.0002,
+      "loss": 0.6208,
+      "step": 610
+    },
+    {
+      "epoch": 4.092409240924092,
+      "grad_norm": 1.0794376134872437,
+      "learning_rate": 0.0002,
+      "loss": 0.5347,
+      "step": 620
+    },
+    {
+      "epoch": 4.158415841584159,
+      "grad_norm": 1.2978978157043457,
+      "learning_rate": 0.0002,
+      "loss": 0.5038,
+      "step": 630
+    },
+    {
+      "epoch": 4.224422442244224,
+      "grad_norm": 1.1555476188659668,
+      "learning_rate": 0.0002,
+      "loss": 0.5086,
+      "step": 640
+    },
+    {
+      "epoch": 4.29042904290429,
+      "grad_norm": 1.2701029777526855,
+      "learning_rate": 0.0002,
+      "loss": 0.515,
+      "step": 650
+    },
+    {
+      "epoch": 4.356435643564357,
+      "grad_norm": 1.2661395072937012,
+      "learning_rate": 0.0002,
+      "loss": 0.5166,
+      "step": 660
+    },
+    {
+      "epoch": 4.422442244224422,
+      "grad_norm": 1.5223569869995117,
+      "learning_rate": 0.0002,
+      "loss": 0.5345,
+      "step": 670
+    },
+    {
+      "epoch": 4.488448844884489,
+      "grad_norm": 1.1259031295776367,
+      "learning_rate": 0.0002,
+      "loss": 0.5093,
+      "step": 680
+    },
+    {
+      "epoch": 4.554455445544555,
+      "grad_norm": 1.1892699003219604,
+      "learning_rate": 0.0002,
+      "loss": 0.5252,
+      "step": 690
+    },
+    {
+      "epoch": 4.62046204620462,
+      "grad_norm": 1.031516671180725,
+      "learning_rate": 0.0002,
+      "loss": 0.5374,
+      "step": 700
+    },
+    {
+      "epoch": 4.686468646864687,
+      "grad_norm": 1.576996922492981,
+      "learning_rate": 0.0002,
+      "loss": 0.5511,
+      "step": 710
+    },
+    {
+      "epoch": 4.752475247524752,
+      "grad_norm": 1.208756923675537,
+      "learning_rate": 0.0002,
+      "loss": 0.5371,
+      "step": 720
+    },
+    {
+      "epoch": 4.818481848184819,
+      "grad_norm": 1.4121363162994385,
+      "learning_rate": 0.0002,
+      "loss": 0.543,
+      "step": 730
+    },
+    {
+      "epoch": 4.884488448844884,
+      "grad_norm": 1.2884345054626465,
+      "learning_rate": 0.0002,
+      "loss": 0.5483,
+      "step": 740
+    },
+    {
+      "epoch": 4.9504950495049505,
+      "grad_norm": 1.3778842687606812,
+      "learning_rate": 0.0002,
+      "loss": 0.5287,
+      "step": 750
+    },
+    {
+      "epoch": 4.996699669966997,
+      "eval_loss": 1.2193516492843628,
+      "eval_runtime": 7.9403,
+      "eval_samples_per_second": 13.727,
+      "eval_steps_per_second": 1.763,
+      "step": 757
+    },
+    {
+      "epoch": 5.016501650165017,
+      "grad_norm": 1.1151881217956543,
+      "learning_rate": 0.0002,
+      "loss": 0.525,
+      "step": 760
+    },
+    {
+      "epoch": 5.082508250825082,
+      "grad_norm": 1.3323009014129639,
+      "learning_rate": 0.0002,
+      "loss": 0.4187,
+      "step": 770
+    },
+    {
+      "epoch": 5.148514851485149,
+      "grad_norm": 1.7906690835952759,
+      "learning_rate": 0.0002,
+      "loss": 0.4083,
+      "step": 780
+    },
+    {
+      "epoch": 5.214521452145214,
+      "grad_norm": 1.215036153793335,
+      "learning_rate": 0.0002,
+      "loss": 0.3952,
+      "step": 790
+    },
+    {
+      "epoch": 5.2805280528052805,
+      "grad_norm": 1.3145633935928345,
+      "learning_rate": 0.0002,
+      "loss": 0.4021,
+      "step": 800
+    },
+    {
+      "epoch": 5.346534653465347,
+      "grad_norm": 1.2486501932144165,
+      "learning_rate": 0.0002,
+      "loss": 0.4073,
+      "step": 810
+    },
+    {
+      "epoch": 5.412541254125412,
+      "grad_norm": 1.5110037326812744,
+      "learning_rate": 0.0002,
+      "loss": 0.4297,
+      "step": 820
+    },
+    {
+      "epoch": 5.478547854785479,
+      "grad_norm": 1.030617594718933,
+      "learning_rate": 0.0002,
+      "loss": 0.418,
+      "step": 830
+    },
+    {
+      "epoch": 5.544554455445544,
+      "grad_norm": 1.6717298030853271,
+      "learning_rate": 0.0002,
+      "loss": 0.4247,
+      "step": 840
+    },
+    {
+      "epoch": 5.6105610561056105,
+      "grad_norm": 1.4030007123947144,
+      "learning_rate": 0.0002,
+      "loss": 0.4286,
+      "step": 850
+    },
+    {
+      "epoch": 5.676567656765677,
+      "grad_norm": 1.288204550743103,
+      "learning_rate": 0.0002,
+      "loss": 0.4262,
+      "step": 860
+    },
+    {
+      "epoch": 5.742574257425742,
+      "grad_norm": 1.0709242820739746,
+      "learning_rate": 0.0002,
+      "loss": 0.4389,
+      "step": 870
+    },
+    {
+      "epoch": 5.808580858085809,
+      "grad_norm": 1.2238759994506836,
+      "learning_rate": 0.0002,
+      "loss": 0.434,
+      "step": 880
+    },
+    {
+      "epoch": 5.874587458745875,
+      "grad_norm": 1.4378324747085571,
+      "learning_rate": 0.0002,
+      "loss": 0.436,
+      "step": 890
+    },
+    {
+      "epoch": 5.9405940594059405,
+      "grad_norm": 1.6783941984176636,
+      "learning_rate": 0.0002,
+      "loss": 0.4282,
+      "step": 900
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.3334790468215942,
+      "eval_runtime": 7.9337,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 909
+    },
+    {
+      "epoch": 6.006600660066007,
+      "grad_norm": 1.2153123617172241,
+      "learning_rate": 0.0002,
+      "loss": 0.4035,
+      "step": 910
+    },
+    {
+      "epoch": 6.072607260726072,
+      "grad_norm": 1.8521552085876465,
+      "learning_rate": 0.0002,
+      "loss": 0.3311,
+      "step": 920
+    },
+    {
+      "epoch": 6.138613861386139,
+      "grad_norm": 1.5133066177368164,
+      "learning_rate": 0.0002,
+      "loss": 0.3199,
+      "step": 930
+    },
+    {
+      "epoch": 6.204620462046204,
+      "grad_norm": 1.724316120147705,
+      "learning_rate": 0.0002,
+      "loss": 0.3108,
+      "step": 940
+    },
+    {
+      "epoch": 6.270627062706271,
+      "grad_norm": 1.2472296953201294,
+      "learning_rate": 0.0002,
+      "loss": 0.3375,
+      "step": 950
+    },
+    {
+      "epoch": 6.336633663366337,
+      "grad_norm": 1.9551066160202026,
+      "learning_rate": 0.0002,
+      "loss": 0.3529,
+      "step": 960
+    },
+    {
+      "epoch": 6.402640264026402,
+      "grad_norm": 1.3545963764190674,
+      "learning_rate": 0.0002,
+      "loss": 0.35,
+      "step": 970
+    },
+    {
+      "epoch": 6.468646864686469,
+      "grad_norm": 1.4807971715927124,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 980
+    },
+    {
+      "epoch": 6.534653465346535,
+      "grad_norm": 1.6042426824569702,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 990
+    },
+    {
+      "epoch": 6.600660066006601,
+      "grad_norm": 1.5707716941833496,
+      "learning_rate": 0.0002,
+      "loss": 0.3632,
+      "step": 1000
+    },
+    {
+      "epoch": 6.666666666666667,
+      "grad_norm": 1.5197912454605103,
+      "learning_rate": 0.0002,
+      "loss": 0.3663,
+      "step": 1010
+    },
+    {
+      "epoch": 6.732673267326732,
+      "grad_norm": 1.9985229969024658,
+      "learning_rate": 0.0002,
+      "loss": 0.3497,
+      "step": 1020
+    },
+    {
+      "epoch": 6.798679867986799,
+      "grad_norm": 1.6729710102081299,
+      "learning_rate": 0.0002,
+      "loss": 0.346,
+      "step": 1030
+    },
+    {
+      "epoch": 6.864686468646864,
+      "grad_norm": 1.3435407876968384,
+      "learning_rate": 0.0002,
+      "loss": 0.3737,
+      "step": 1040
+    },
+    {
+      "epoch": 6.930693069306931,
+      "grad_norm": 1.4809728860855103,
+      "learning_rate": 0.0002,
+      "loss": 0.3646,
+      "step": 1050
+    },
+    {
+      "epoch": 6.996699669966997,
+      "grad_norm": 1.7941899299621582,
+      "learning_rate": 0.0002,
+      "loss": 0.3519,
+      "step": 1060
+    },
+    {
+      "epoch": 6.996699669966997,
+      "eval_loss": 1.4348182678222656,
+      "eval_runtime": 7.9375,
+      "eval_samples_per_second": 13.732,
+      "eval_steps_per_second": 1.764,
+      "step": 1060
+    },
+    {
+      "epoch": 7.0627062706270625,
+      "grad_norm": 1.5893484354019165,
+      "learning_rate": 0.0002,
+      "loss": 0.2818,
+      "step": 1070
+    },
+    {
+      "epoch": 7.128712871287129,
+      "grad_norm": 1.3127269744873047,
+      "learning_rate": 0.0002,
+      "loss": 0.2677,
+      "step": 1080
+    },
+    {
+      "epoch": 7.194719471947194,
+      "grad_norm": 1.4710880517959595,
+      "learning_rate": 0.0002,
+      "loss": 0.2693,
+      "step": 1090
+    },
+    {
+      "epoch": 7.260726072607261,
+      "grad_norm": 1.7166521549224854,
+      "learning_rate": 0.0002,
+      "loss": 0.2851,
+      "step": 1100
+    },
+    {
+      "epoch": 7.326732673267327,
+      "grad_norm": 1.8188034296035767,
+      "learning_rate": 0.0002,
+      "loss": 0.2816,
+      "step": 1110
+    },
+    {
+      "epoch": 7.3927392739273925,
+      "grad_norm": 1.5059692859649658,
+      "learning_rate": 0.0002,
+      "loss": 0.2856,
+      "step": 1120
+    },
+    {
+      "epoch": 7.458745874587459,
+      "grad_norm": 1.3958371877670288,
+      "learning_rate": 0.0002,
+      "loss": 0.2948,
+      "step": 1130
+    },
+    {
+      "epoch": 7.524752475247524,
+      "grad_norm": 1.4339087009429932,
+      "learning_rate": 0.0002,
+      "loss": 0.2832,
+      "step": 1140
+    },
+    {
+      "epoch": 7.590759075907591,
+      "grad_norm": 1.5262160301208496,
+      "learning_rate": 0.0002,
+      "loss": 0.3095,
+      "step": 1150
+    },
+    {
+      "epoch": 7.656765676567657,
+      "grad_norm": 1.704048991203308,
+      "learning_rate": 0.0002,
+      "loss": 0.2741,
+      "step": 1160
+    },
+    {
+      "epoch": 7.7227722772277225,
+      "grad_norm": 1.6189782619476318,
+      "learning_rate": 0.0002,
+      "loss": 0.2986,
+      "step": 1170
+    },
+    {
+      "epoch": 7.788778877887789,
+      "grad_norm": 1.4849400520324707,
+      "learning_rate": 0.0002,
+      "loss": 0.2946,
+      "step": 1180
+    },
+    {
+      "epoch": 7.854785478547855,
+      "grad_norm": 1.9224109649658203,
+      "learning_rate": 0.0002,
+      "loss": 0.2886,
+      "step": 1190
+    },
+    {
+      "epoch": 7.920792079207921,
+      "grad_norm": 1.4852501153945923,
+      "learning_rate": 0.0002,
+      "loss": 0.3072,
+      "step": 1200
+    },
+    {
+      "epoch": 7.973597359735973,
+      "eval_loss": 1.4857953786849976,
+      "eval_runtime": 7.9364,
+      "eval_samples_per_second": 13.734,
+      "eval_steps_per_second": 1.764,
+      "step": 1208
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.30006698622976e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-1208/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2de0dfe312263bfc3e8d6e80fdd97c7a6f0bdac64594686d024b1965cb27e1a6
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0517f5c8411a81ef867ebc058c15b87fd1d3e9fa7b83baef613577132dd183b
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:105a92a702b72223b5e43b3cb09bcff0aec5a744895d108da7002453bcdfab74
+size 55532538

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:711075c28237eacc85afe86ce07498f8ae8e5c9ac31a50964217c1f49396635b
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c07c30f6492233125a9dcc979e89ca22c8915eafad3395b418390309021408d8
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/trainer_state.json

@@ -0,0 +1,146 @@
+{
+  "best_metric": 1.0424563884735107,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151",
+  "epoch": 0.9966996699669967,
+  "eval_steps": 10,
+  "global_step": 151,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.066006600660066,
+      "grad_norm": 0.8332476615905762,
+      "learning_rate": 0.0002,
+      "loss": 1.6257,
+      "step": 10
+    },
+    {
+      "epoch": 0.132013201320132,
+      "grad_norm": 1.1134452819824219,
+      "learning_rate": 0.0002,
+      "loss": 1.3155,
+      "step": 20
+    },
+    {
+      "epoch": 0.19801980198019803,
+      "grad_norm": 0.8637537956237793,
+      "learning_rate": 0.0002,
+      "loss": 1.2979,
+      "step": 30
+    },
+    {
+      "epoch": 0.264026402640264,
+      "grad_norm": 1.291804313659668,
+      "learning_rate": 0.0002,
+      "loss": 1.1495,
+      "step": 40
+    },
+    {
+      "epoch": 0.33003300330033003,
+      "grad_norm": 0.6008025407791138,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 50
+    },
+    {
+      "epoch": 0.39603960396039606,
+      "grad_norm": 0.6817359924316406,
+      "learning_rate": 0.0002,
+      "loss": 1.0974,
+      "step": 60
+    },
+    {
+      "epoch": 0.46204620462046203,
+      "grad_norm": 0.7527281641960144,
+      "learning_rate": 0.0002,
+      "loss": 1.0563,
+      "step": 70
+    },
+    {
+      "epoch": 0.528052805280528,
+      "grad_norm": 0.6140271425247192,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 80
+    },
+    {
+      "epoch": 0.594059405940594,
+      "grad_norm": 0.6146464943885803,
+      "learning_rate": 0.0002,
+      "loss": 1.0977,
+      "step": 90
+    },
+    {
+      "epoch": 0.6600660066006601,
+      "grad_norm": 0.6727924942970276,
+      "learning_rate": 0.0002,
+      "loss": 1.1484,
+      "step": 100
+    },
+    {
+      "epoch": 0.7260726072607261,
+      "grad_norm": 0.6649640798568726,
+      "learning_rate": 0.0002,
+      "loss": 1.0635,
+      "step": 110
+    },
+    {
+      "epoch": 0.7920792079207921,
+      "grad_norm": 0.5999083518981934,
+      "learning_rate": 0.0002,
+      "loss": 1.0545,
+      "step": 120
+    },
+    {
+      "epoch": 0.858085808580858,
+      "grad_norm": 0.638982355594635,
+      "learning_rate": 0.0002,
+      "loss": 1.0681,
+      "step": 130
+    },
+    {
+      "epoch": 0.9240924092409241,
+      "grad_norm": 0.604159951210022,
+      "learning_rate": 0.0002,
+      "loss": 1.0369,
+      "step": 140
+    },
+    {
+      "epoch": 0.9900990099009901,
+      "grad_norm": 0.7679715752601624,
+      "learning_rate": 0.0002,
+      "loss": 1.0963,
+      "step": 150
+    },
+    {
+      "epoch": 0.9966996699669967,
+      "eval_loss": 1.0424563884735107,
+      "eval_runtime": 7.9324,
+      "eval_samples_per_second": 13.741,
+      "eval_steps_per_second": 1.765,
+      "step": 151
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6647021096140800.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-151/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2de0dfe312263bfc3e8d6e80fdd97c7a6f0bdac64594686d024b1965cb27e1a6
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5690252b01a9287a242580edaed9ff30af1008823786e04a65b411b59ecfdc44
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:947735509933125536218e67de41fc2a212863466978fb611b9289d0bc19ee7c
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e42085dd0f7da10465be73dccab9891ba8f6ad5e3cae4a68d2d40471754c75ed
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79d838ec0f54330713e95cf2267f891e8ebdc900ea52692c0859c1b1866ced98
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303/trainer_state.json

@@ -0,0 +1,259 @@
+{
+  "best_metric": 1.0289618968963623,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-42/checkpoint-303",
+  "epoch": 2.0,
+  "eval_steps": 10,
+  "global_step": 303,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.066006600660066,
+      "grad_norm": 0.8332476615905762,
+      "learning_rate": 0.0002,
+      "loss": 1.6257,
+      "step": 10
+    },
+    {
+      "epoch": 0.132013201320132,
+      "grad_norm": 1.1134452819824219,
+      "learning_rate": 0.0002,
+      "loss": 1.3155,
+      "step": 20
+    },
+    {
+      "epoch": 0.19801980198019803,
+      "grad_norm": 0.8637537956237793,
+      "learning_rate": 0.0002,
+      "loss": 1.2979,
+      "step": 30
+    },
+    {
+      "epoch": 0.264026402640264,
+      "grad_norm": 1.291804313659668,
+      "learning_rate": 0.0002,
+      "loss": 1.1495,
+      "step": 40
+    },
+    {
+      "epoch": 0.33003300330033003,
+      "grad_norm": 0.6008025407791138,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 50
+    },
+    {
+      "epoch": 0.39603960396039606,
+      "grad_norm": 0.6817359924316406,
+      "learning_rate": 0.0002,
+      "loss": 1.0974,
+      "step": 60
+    },
+    {
+      "epoch": 0.46204620462046203,
+      "grad_norm": 0.7527281641960144,
+      "learning_rate": 0.0002,
+      "loss": 1.0563,
+      "step": 70
+    },
+    {
+      "epoch": 0.528052805280528,
+      "grad_norm": 0.6140271425247192,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 80
+    },
+    {
+      "epoch": 0.594059405940594,
+      "grad_norm": 0.6146464943885803,
+      "learning_rate": 0.0002,
+      "loss": 1.0977,
+      "step": 90
+    },
+    {
+      "epoch": 0.6600660066006601,
+      "grad_norm": 0.6727924942970276,
+      "learning_rate": 0.0002,
+      "loss": 1.1484,
+      "step": 100
+    },
+    {
+      "epoch": 0.7260726072607261,
+      "grad_norm": 0.6649640798568726,
+      "learning_rate": 0.0002,
+      "loss": 1.0635,
+      "step": 110
+    },
+    {
+      "epoch": 0.7920792079207921,
+      "grad_norm": 0.5999083518981934,
+      "learning_rate": 0.0002,
+      "loss": 1.0545,
+      "step": 120
+    },
+    {
+      "epoch": 0.858085808580858,
+      "grad_norm": 0.638982355594635,
+      "learning_rate": 0.0002,
+      "loss": 1.0681,
+      "step": 130
+    },
+    {
+      "epoch": 0.9240924092409241,
+      "grad_norm": 0.604159951210022,
+      "learning_rate": 0.0002,
+      "loss": 1.0369,
+      "step": 140
+    },
+    {
+      "epoch": 0.9900990099009901,
+      "grad_norm": 0.7679715752601624,
+      "learning_rate": 0.0002,
+      "loss": 1.0963,
+      "step": 150
+    },
+    {
+      "epoch": 0.9966996699669967,
+      "eval_loss": 1.0424563884735107,
+      "eval_runtime": 7.9324,
+      "eval_samples_per_second": 13.741,
+      "eval_steps_per_second": 1.765,
+      "step": 151
+    },
+    {
+      "epoch": 1.056105610561056,
+      "grad_norm": 0.9450559616088867,
+      "learning_rate": 0.0002,
+      "loss": 0.9183,
+      "step": 160
+    },
+    {
+      "epoch": 1.1221122112211221,
+      "grad_norm": 0.6976049542427063,
+      "learning_rate": 0.0002,
+      "loss": 0.9742,
+      "step": 170
+    },
+    {
+      "epoch": 1.188118811881188,
+      "grad_norm": 0.6649869680404663,
+      "learning_rate": 0.0002,
+      "loss": 0.9278,
+      "step": 180
+    },
+    {
+      "epoch": 1.2541254125412542,
+      "grad_norm": 0.7072765827178955,
+      "learning_rate": 0.0002,
+      "loss": 0.961,
+      "step": 190
+    },
+    {
+      "epoch": 1.3201320132013201,
+      "grad_norm": 0.7633873224258423,
+      "learning_rate": 0.0002,
+      "loss": 0.9627,
+      "step": 200
+    },
+    {
+      "epoch": 1.386138613861386,
+      "grad_norm": 0.7836750149726868,
+      "learning_rate": 0.0002,
+      "loss": 0.9528,
+      "step": 210
+    },
+    {
+      "epoch": 1.4521452145214522,
+      "grad_norm": 0.6136282682418823,
+      "learning_rate": 0.0002,
+      "loss": 0.9099,
+      "step": 220
+    },
+    {
+      "epoch": 1.5181518151815183,
+      "grad_norm": 0.6747711300849915,
+      "learning_rate": 0.0002,
+      "loss": 0.9719,
+      "step": 230
+    },
+    {
+      "epoch": 1.5841584158415842,
+      "grad_norm": 0.8176482319831848,
+      "learning_rate": 0.0002,
+      "loss": 0.953,
+      "step": 240
+    },
+    {
+      "epoch": 1.6501650165016502,
+      "grad_norm": 0.5480154156684875,
+      "learning_rate": 0.0002,
+      "loss": 0.9546,
+      "step": 250
+    },
+    {
+      "epoch": 1.716171617161716,
+      "grad_norm": 0.6764057874679565,
+      "learning_rate": 0.0002,
+      "loss": 0.932,
+      "step": 260
+    },
+    {
+      "epoch": 1.7821782178217822,
+      "grad_norm": 0.8389566540718079,
+      "learning_rate": 0.0002,
+      "loss": 1.0034,
+      "step": 270
+    },
+    {
+      "epoch": 1.8481848184818483,
+      "grad_norm": 0.6570506691932678,
+      "learning_rate": 0.0002,
+      "loss": 0.9758,
+      "step": 280
+    },
+    {
+      "epoch": 1.9141914191419143,
+      "grad_norm": 0.7700225710868835,
+      "learning_rate": 0.0002,
+      "loss": 1.023,
+      "step": 290
+    },
+    {
+      "epoch": 1.9801980198019802,
+      "grad_norm": 0.6293354630470276,
+      "learning_rate": 0.0002,
+      "loss": 0.971,
+      "step": 300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0289618968963623,
+      "eval_runtime": 7.9336,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 303
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.32940421922816e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}