Experiments with Tree-Diffusion: Testing a New Code Generation Method on Graphics Languages

Table of Links

Abstract and 1. Introduction

2. Background & Related Work

3. Method

   3.1 Sampling Small Mutations

   3.2 Policy

   3.3 Value Network & Search

   3.4 Architecture

4. Experiments

   4.1 Environments

   4.2 Baselines

   4.3 Ablations

5. Conclusion, Acknowledgments and Disclosure of Funding, and References

Appendix

A. Mutation Algorithm

B. Context-Free Grammars

C. Sketch Simulation

D. Complexity Filtering

E. Tree Path Algorithm

F. Implementation Details

4 Experiments

4.1 Environments

We conduct experiments on four domain-specific graphics languages, with complete grammar specifications provided in Appendix B.

CSG2D A 2D constructive solid geometry language where primitive shapes are added and subtracted to create more complex forms, as explored in our baseline methods [11, 28]. We also create CSG2D-Sketch, which has an added observation model that simulates hand-drawn sketches using the algorithm from Wood et al. [39].

TinySVG A language featuring primitive shapes with color, along with Arrange commands for horizontal and vertical alignment, and Move commands for shape offsetting. Figure 1 portrays an example program. Unlike the compositional nature of CSG2D, TinySVG is hierarchical: subexpressions can be combined into compound objects for high-level manipulation. We also create, Rainbow, a simplified version of TinySVG without Move commands for ablation studies due to its reduced computational demands.

We implemented these languages using the Lark [19] and Iceberg [16] Python libraries, with our tree-diffusion implementation designed to be generic and adaptable to any context-free grammar and observation model.