Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -144,10 +144,6 @@ Aurora provides the **largest gains at small-to-moderate batch sizes**, with up
|
|
| 144 |
- **Speedup**: Relative to the no-speculation baseline
|
| 145 |
- **P05/P95**: 5th and 95th percentile throughput values
|
| 146 |
|
| 147 |
-
### Future Work
|
| 148 |
-
|
| 149 |
-
While Aurora demonstrates strong performance at small-to-moderate batch sizes, there is room for improvement in high-batch scenarios. Future work includes making speculative decoding faster for hybrid models, potentially by using **multistep kernels** that can more efficiently handle the verification overhead in batched settings.
|
| 150 |
-
|
| 151 |
**Notably**, this performance is achieved with a model trained **from scratch** - it learns entirely through Aurora's online training process, demonstrating the effectiveness of inference-time training without expensive pre-training.
|
| 152 |
|
| 153 |
## Usage
|
|
|
|
| 144 |
- **Speedup**: Relative to the no-speculation baseline
|
| 145 |
- **P05/P95**: 5th and 95th percentile throughput values
|
| 146 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
**Notably**, this performance is achieved with a model trained **from scratch** - it learns entirely through Aurora's online training process, demonstrating the effectiveness of inference-time training without expensive pre-training.
|
| 148 |
|
| 149 |
## Usage
|