# Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models

Authors: Yushi Hu, Otilia Stretcu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, Ariel Fuxman.

Trains a vision-language model on answers and reasoning steps from executed visual programs. Separate answer and rationale objectives let the model answer image questions without generating the intermediate program or rationale at inference.

First public: 2023-12-05.
ArXiv initial submission: 2023-12-05.

[Original paper](https://arxiv.org/abs/2312.03052) · [PDF](https://arxiv.org/pdf/2312.03052) · [HTML](https://arxiv.org/html/2312.03052v2) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2312.03052/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2312.03052/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2312.03052/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2312.03052/record.json)

## Published version

Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models. Yushi Hu, Otilia Stretcu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, Ariel Fuxman. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[Publication](https://openaccess.thecvf.com/content/CVPR2024/html/Hu_Visual_Program_Distillation_Distilling_Tools_and_Programmatic_Reasoning_into_Vision-Language_CVPR_2024_paper.html) · [BibTeX](https://latent-implicit-thinking.github.io/papers/2312.03052/publication.bib) · [CSL-JSON](https://latent-implicit-thinking.github.io/papers/2312.03052/publication.csl.json)
[Publication record](https://openaccess.thecvf.com/content/CVPR2024/html/Hu_Visual_Program_Distillation_Distilling_Tools_and_Programmatic_Reasoning_into_Vision-Language_CVPR_2024_paper.html).

## Approaches

- [Internalized reasoning](https://latent-implicit-thinking.github.io/topics/internalized-reasoning/): Learn from worked solutions so that some or all of the reasoning can later happen without writing those steps. Training may use a separate teacher model or progressively remove written steps from the model being trained.

Reading-list version 0.11.0; literature checked through 2026-10-26.