Reinforcement Learning
rl4co
File size: 4,300 Bytes
0498a46
 
4edf73d
 
0498a46
 
4edf73d
0498a46
4edf73d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
---
license: mit
pipeline_tag: reinforcement-learning
library_name: rl4co
---

# RouteFinder: Towards Foundation Models for Vehicle Routing Problems

This repository contains the checkpoints for **RouteFinder**, a comprehensive foundation model framework designed to tackle various Vehicle Routing Problem (VRP) variants. This model was presented in the paper [RouteFinder: Towards Foundation Models for Vehicle Routing Problems](https://huggingface.co/papers/2406.15007).

The official code and detailed instructions are available in the [GitHub repository](https://github.com/ai4co/routefinder).

## Abstract

This paper introduces RouteFinder, a comprehensive foundation model framework to tackle different Vehicle Routing Problem (VRP) variants. Our core idea is that a foundation model for VRPs should be able to represent variants by treating each as a subset of a generalized problem equipped with different attributes. We propose a unified VRP environment capable of efficiently handling any combination of these attributes. The RouteFinder model leverages a modern transformer-based encoder and global attribute embeddings to improve task representation. Additionally, we introduce two reinforcement learning techniques to enhance multi-task performance: mixed batch training, which enables training on different variants at once, and multi-variant reward normalization to balance different reward scales. Finally, we propose efficient adapter layers that enable fine-tuning for new variants with unseen attributes. Extensive experiments on 48 VRP variants show RouteFinder outperforms recent state-of-the-art learning methods. Our code is publicly available at this https URL .

## Installation

We use [uv](https://github.com/astral-sh/uv) (Python package manager) to manage the dependencies:

```bash
uv venv --python 3.12 # create a new virtual environment
source .venv/bin/activate # activate the virtual environment
uv sync --all-extras # for all dependencies
```

Note that this project is also compatible with normal `pip install -e .` in case you use a different package manager.

## Quickstart

### Download data and checkpoints

To download the data and checkpoints from HuggingFace automatically, you can use:

```bash
python scripts/download_hf.py
```

### Running

We recommend exploring [this quickstart notebook](https://github.com/ai4co/routefinder/blob/main/examples/1.quickstart.ipynb) to get started with the `RouteFinder` codebase!

The main runner (example here of main baseline) can be called via:

```bash
python run.py experiment=main/rf/rf-transformer-100
```
You may change the experiment by using the `experiment=YOUR_EXP`, with the path under [`configs/experiment`](https://github.com/ai4co/routefinder/tree/main/configs/experiment) directory.

### Testing

You may use the provided test function to test the model:

```bash
python test.py --checkpoint checkpoints/100/rf-transformer.ckpt
```

or with additional parameters:

```
usage: test.py [-h] --checkpoint CHECKPOINT [--problem PROBLEM] [--size SIZE] [--datasets DATASETS] [--batch_size BATCH_SIZE]
               [--device DEVICE] [--remove-mixed-backhaul | --no-remove-mixed-backhaul]

options:
  -h, --help            show this help message and exit
  --checkpoint CHECKPOINT
                        Path to the model checkpoint
  --problem PROBLEM     Problem name: cvrp, vrptw, etc. or all
  --size SIZE           Problem size: 50, 100, for automatic loading
  --datasets DATASETS   Filename of the dataset(s) to evaluate. Defaults to all under data/{problem}/ dir
  --batch_size BATCH_SIZE
  --device DEVICE
  --remove-mixed-backhaul, --no-remove-mixed-backhaul
                        Remove mixed backhaul instances. Use --no-remove-mixed-backhaul to keep them. (default: True)
```

## Citation

If you find RouteFinder valuable for your research or applied projects:

```bibtex
@article{
berto2025routefinder,
title={{RouteFinder: Towards Foundation Models for Vehicle Routing Problems}},
author={Federico Berto and Chuanbo Hua and Nayeli Gast Zepeda and Andr{\'e} Hottung and Niels Wouda and Leon Lan and Junyoung Park and Kevin Tierney and Jinkyoo Park},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2025},
url={https://openreview.net/forum?id=QzGLoaOPiY},
}
```