Inputs z and p
types stay put, positions turn
Atomic numbers z and 3D positions p are all the network is given. Everything else in the drawing is computed from them.
Section 3, Figure 2(a), page 4Geometric deep learning ICLR 2025
Turn the molecule. Some numbers turn with it. The rest never move.
GotenNet: Rethinking Efficient 3D Equivariant Graph Neural Networks
The paper proposes a network whose atoms carry direction-aware features up to degree 2, combines those features through inner products in place of Clebsch-Gordan tensor products, and reports lower errors than the models it compares against on most targets of four molecular benchmarks.
Figure 1Illustration
Turns with the molecule
Direction, atom 2 to atom 1
x−0.990
y−0.119
z−0.079
Degree-2 feature, same bond
10.166
20.013
3−0.401
40.111
50.683
Stays put
Distance, atoms 1 and 21.263
Inner product of the two bonds, degree 1−0.317
Inner product of the two bonds, degree 2−0.233
You can also use the slider and the Spin button that follow.
01 The problem
The paper starts from a choice: cheap geometry or rich geometry.
The paper wants a network that is strictly equivariant: turn or move the molecule and its features follow. It sorts earlier equivariant models into two families with opposite strengths, then places its own design between them.
Scalarization-based: project to scalars, rebuild directions
- What the paper credits it with
- “offering computational efficiency and scalability for large-scale applications”
- What the paper says limits it
- “this projection process may limit expressiveness in capturing complex geometric patterns”
Rows the paper files here in Table 9: ClofNet, EGNN, PaiNN†, LEFTNet, EQGAT, ET, Geoformer, SaVeNet-B†.
Source: Section 1, page 2; Table 9, page 23High-degree steerable: stacks of features, tensor products
- What the paper credits it with
- “achieve impressive performance through irreducible representations (irreps) and Clebsch-Gordan (CG) transforms”
- What the paper says limits it
- “these models incur significant computational overhead due to their reliance on complex tensor operations”
Rows the paper files under this heading in Table 9: SEGNN, HDGNN†, Equiformer, EquiformerV2. The box is left closed on purpose: this is how the paper describes the family (Section 2.2). It does not explain the operation or put a cost on it, and neither does this page.
Source: Section 1, page 2; Section 2.2, page 3; Table 9, page 23GotenNet: the paper calls it spherical-scalarization
- The paper’s own label
- “a spherical-scalarization model with an efficient representation and embedding strategy designed specifically with geometric tensors, eliminating the need for irreps and CG transforms”
- Whose idea the inner product is
- The paper credits it to earlier work (it cites ViSNet, SO3KRATES and HEGNN). Its own step, in its words: “geometry-aware tensor attention, which employs a concise formulation of inner product operations combined with hierarchical refinement mechanisms”.
Rows under “Spherical-scalarization models” in Table 9: GotenNet S, B and L.
Source: Section 1, page 2; Section 2.2, pages 3 to 4; Table 9, page 23Figure 2Measured, Table 9
Pick a family above and find it on one axis
- invariant
- scalarization-based
- high-degree steerable
- pre-trained
- GotenNet (this paper)
With scripts on, this figure places every row of Table 9 on one axis of standardized MAE. The same values are in the QM9 table in the evidence section.
02 The idea
Directions are only ever scaled and added. When two of them meet, the result is a plain number.
The paper’s network keeps two kinds of feature on every atom: invariant ones and steerable ones. Below is a hand-built toy with the same algebra. Turn it and watch which column moves. Attention sits in the column that does not.
Figure 3Illustration
Turn the toy, choose a pair of atoms, then break the rule
- arrow: an atom’s degree-1 feature
- double lines: its degree-2 feature
- wedges: attention of atom 1
The sliders, selects and buttons that follow do everything the pointer does.
Numbers for atom 1 listening to atom 2, after 2 blocks
Turns with the molecule
Edge direction
x−0.990
y−0.119
z−0.079
Degree-1 feature of atom 1
x−1.120
y−0.389
z−0.175
Degree-2 feature of atom 1
10.537
2−0.256
3−0.794
4−0.277
50.734
Five numbers that turn together. The paper computes its degree-2 features from spherical harmonics; this toy stores the same kind of object in its own basis.
Stays put
Distance1.263
Scalar feature of atom 11.268
Inner product, degree 10.160
Inner product, degree 20.630
Refined edge value2.154
Attention, atom 1 on 20.566
Length, degree-1 feature1.199
Length, degree-2 feature1.265
Largest change in this column since the start
0
Rule intact: one weight acts on x, y and z alike. However you hold the molecule, this column holds still to within floating-point residue.
Move it
The pair
The counterfactual
This broken variant is ours. The paper states the rule (Section 3.4) and does not test breaking it.
IllustrationA hand-built toy with five made-up atoms, one feature channel and fixed numbers in place of learned weights. It shows which quantities turn with the molecule and which do not. It is not the trained network and its numbers are not results.
- Drag the molecule or move a slider. The left column moves with it. The right column, attention included, does not.
- Press Shift position. Neither column changes, because every number here is built from differences between positions.
- Switch on Break the rule with 2 or more blocks, then turn the molecule. The right column starts to move.
Degree 1 only matches the one other setting the paper tries. No accuracy is claimed here. The measured effect is one ablation row on QM9 at six layers: Lmax = 1 prints a std. MAE of 0.68 against 0.61 for Lmax = 2 (Table 5, page 10).
A number that stays put, times a feature that turns, still turns
This is how every steerable update in the paper is built: coefficients computed from invariants scale the edge’s direction features and the neighbour’s features, and the results are summed.
Equation 8, page 6; Appendix B, page 17Two turning features of the same degree meet only in an inner product
The rotation cancels, so the result stays put. The paper uses it to send geometric detail back into the edge, which then shapes attention in the next block.
Equations 10 to 12, page 7; Appendix B, page 18One weight must act on every component of a degree
The paper’s words: “To preserve equivariance, uniform weights are applied across spatial dimensions.” Break the rule above to see what it protects.
Section 3.4, page 7; Appendix B, pages 17 to 1803 One block, step by step
Follow one pair of atoms through one block, then watch the edge sharpen across blocks.
The drawing has two lanes. The upper lane holds numbers that stay put, the lower lane features that turn. Watch where the lanes cross: a multiply going down, an inner product and a length going up. The live values are the toy’s, for the pair and the orientation you left in the turning test.
Figure 4Illustration
Step through the block, or select any part of the drawing
Single lines carry numbers that stay put. Double lines carry features that turn. Our reading of Equations 8 to 13: no step combines two turning features into a new turning feature, which is why that box is struck out. The paper’s own words: “without relying on irreducible representations or Clebsch-Gordan transforms”. This is an original drawing made from the equations, not the paper’s Figure 2.
-
Two atoms and the line between them
The edge from atom j to atom i is described at three levels: how far (1 number), which way (3 numbers), and a degree-2 feature (5 numbers). The paper uses levels 0 to Lmax = 2.
- Distance
- 1.263
- Direction
- −0.990, −0.119, −0.079
- Degree 2
- 0.166, 0.013, −0.401, 0.111, 0.683
-
What each atom knows before geometry flows
Each atom sums its neighbours’ atom types, weighted by distance and a cutoff, and mixes in its own type. Each edge combines the two atoms’ scalars with the distance. Everything so far stays put when the molecule turns.
- h, atom i
- 1.299
- h, atom j
- 1.311
- t, start
- 1.363
-
The edge asks how the two atoms’ surroundings line up
Hierarchical Tensor Refinement: for each degree, take the inner product of atom i’s and atom j’s steerable features, combine across degrees, and add the result into the edge scalar. Two turning inputs, one output that stays put.
- Degree 1
- 0.160
- Degree 2
- 0.630
- t before
- 1.363
- t after
- 2.154
In the toy’s first block there is nothing to compare yet, so both inner products are 0. The paper initialises the steerable features during its first interaction layer (Equation 4) and does not describe this case further.
Equations 10 to 12, page 7; Appendix B, page 18 -
Attention decides whom to listen to, using only numbers that stay put
A query from atom i, a key from atom j, and the key is multiplied element-wise by a transform of the edge scalar before the dot product. A softmax runs over the neighbours of atom i. The paper calls this a geometry-infused key.
- Atom i on
- atom 2: 57%, atom 3: 14%, atom 4: 19%, atom 5: 10%
-
One message becomes five sets of coefficients
The attention output plus a geometric-encoding term is split into S = 1 + 2 × Lmax = 5 blocks: one for the scalar update and, for each degree, one that will scale the edge direction and one that will scale the neighbour’s feature.
- Toy’s five
- s 0.108, d1 1.085, d2 1.085, t1 0.325, t2 0.325
In the toy the five are one message times three fixed numbers shared by both degrees, which is closer in spirit to the paper’s shared-coefficient variant than to its headline one.
Equation 7, page 6; Appendix F, page 21 -
Coefficients scale directions. Nothing else touches them.
Atom i’s new degree-l feature is a sum over neighbours of a coefficient times the edge’s direction feature, plus a coefficient times the neighbour’s feature. Scalars get their own sum. Both are added back as residuals.
- Change, h
- 0.196
- Change, deg 1
- −0.975, −0.350, −0.135
- Change, deg 2
- 0.574, −0.344, −0.797, −0.330, 0.665
-
A gate made from lengths, then the block repeats
The equivariant feed-forward takes the length of each steerable feature, a number that stays put, mixes it with h, and produces one update for h and one multiplier for the steerable feature. The refined edge feeds the next block’s attention. On QM9 the paper’s S, B and L sizes have 4, 6 and 12 layers. See the edge change block by block.
- Length, deg 1
- 1.977
- Length, deg 2
- 2.086
- Gate
- 0.743
- Multiplier
- 0.606
Select any part of the drawing to read what it is in the paper. Arrow keys move between parts.
Distance, the degree-0 part of the edge
stays put
The paper: it “captures the scalar distance between nodes, providing rotation and translation invariant information”. Toy value for this pair: 1.263.
Section 3.1, page 4Direction, the degree-1 part of the edge
turns
The unit vector from atom j to atom i, (pi − pj) divided by its length. The paper says it “encodes directional information”. Toy value: −0.990, −0.119, −0.079.
Section 3.1, page 5The degree-2 part of the edge
turns
For degree l of 2 and above the paper uses “(2l + 1) functions derived from spherical harmonics of degree l”, computed with the e3nn library. The toy stores the same kind of object in its own basis, so its five numbers are not in the paper’s order or scale: 0.166, 0.013, −0.401, 0.111, 0.683.
Section 3.1, page 5; Appendix E, page 20Atom scalar features h
stay put
Each atom sums messages from its neighbours: the neighbour’s atom-type embedding times a projection of radial basis functions of the distance and a cutoff. The sum is joined with the atom’s own embedding and projected. Toy values: atom i 1.299, atom j 1.311.
Equations 1 and 2, page 5Edge scalar features t
stay put
Initialised from the two atoms’ scalars and the distance (Equation 3), then refined in every block by the inner-product step (Equation 12). Toy value: 1.363 before this block’s refinement, 2.154 after.
Equation 3, page 6; Equation 12, page 7High-degree steerable features X
turn
The paper: a “feature of degree l is represented by exactly 1 + 2l components”, for degrees 1 to Lmax. Toy values for atom i: degree 1 −1.120, −0.389, −0.175; degree 2 0.537, −0.256, −0.794, −0.277, 0.734.
Section 3.1, page 5Inner product: Hierarchical Tensor Refinement
two turning inputs, one output that stays put
A query projection shared across degrees and a key projection per degree, with “uniform weights” across spatial dimensions, then an inner product per degree, aggregated over degrees. Toy values: degree 1 0.160, degree 2 0.630.
Equations 10 and 11, page 7; Appendix B, pages 17 to 18Attention: geometry-aware tensor attention
stays put
The query and key come from h, and the key is modulated by the edge scalar, so every input is an invariant. Table 6 lists 8 attention heads. The equations, and this toy, show one. Toy weights of atom i: atom 2: 57%, atom 3: 14%, atom 4: 19%, atom 5: 10%.
Equations 5 and 6, page 6; Table 6, page 20Coefficients: the split
stay put
The attention output plus what the paper calls the geometric encoding is split into S = 5 blocks when Lmax = 2. The dashed line is the cutoff of the distance, which enters that term. Toy values: s 0.108, d1 1.085, d2 1.085, t1 0.325, t2 0.325.
Equation 7, page 6Multiply: a coefficient scales a turning feature
a number that stays put, times a feature that turns
The appendix shows that an invariant scalar times an equivariant tensor is still equivariant. Every steerable update in the paper has this form. Toy change to atom i’s degree-1 feature in this block: −0.975, −0.350, −0.135.
Equation 8, page 6; Appendix B, page 17Updated steerable features
turn
In the paper’s update every degree from 1 to Lmax contributes its own component, weighted by its own pair of coefficients, and the result is added back as a residual. Toy value, atom i, degree 1: −1.120, −0.389, −0.175.
Equations 8 and 9, page 6Length of a steerable feature
one turning input, one output that stays put
The L2 norm of a steerable feature does not change when the molecule turns or moves. Toy values for atom i: degree 1 1.977, degree 2 2.086.
Equation 13, page 7; Appendix B, page 18Gate: the equivariant feed-forward
stays put
The lengths are joined with h and passed through a learned function whose output is split in two: one part updates h, the other multiplies the steerable feature. Toy values: gate 0.743, multiplier 0.606.
Equation 13, Section 3.5, page 7One block, repeated
The refined edge enters the next block twice: in the key of the attention and in the geometric encoding. For QM9 the paper varies the number of layers: 4, 6 and 12 for its S, B and L sizes. The toy’s blocks are its own and are not the paper’s layer count.
Figure 2(a) and 2(b), page 4; Appendix E, page 21Decoder
The paper: “the final layer producing either equivariant geometric features or invariant representations as required by the task”. It does not describe the decoder’s internals, so this page does not either.
Section 3.1, page 5Figure 5Illustration
Run more blocks, and switch the edge refinement off
Hierarchical tensor refinement
With scripts on, this figure shows the toy’s edge value and attention block by block, with the refinement on or off.
IllustrationToy numbers. Whether refinement helps accuracy is a measured question: in the paper’s QM9 ablation the six-layer model prints a std. MAE of 0.61 with HTR and 0.64 without it (Table 5, page 10).
04 The evidence
On four benchmarks the paper prints the lowest error in most columns, and the tables show where it does not.
Every value below is a printed cell. Bars start at zero and lower is better. Which cell is lowest, and every count, is computed by this page from the cells, so it can differ from the paper’s own bold and underline marks.
QM9: 12 targets, Tables 1 and 9
Measured, Tables 1 and 9Twelve regression targets, which the paper lists by symbol only, plus a standardized MAE and a logarithmic MAE across all targets. Mean absolute error, lower is better.
The full table: 26 rows as printed in Table 9
| Model | α | Δε | εHOMO | εLUMO | μ | Cν | G | H | R2 | U | U0 | ZPVE | std. | log |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Invariant models | ||||||||||||||
| Cormorant | 85 | 61 | 34 | 38 | 38 | 26 | 20 | 21 | 961 | 21 | 22 | 2.03 | 2.14 | -4.75 |
| NMP | 92 | 69 | 43 | 38 | 30 | 40 | 19 | 17 | 180 | 20 | 20 | 1.50 | 1.78 | -5.08 |
| DimeNet++† | 44 | 32.6 | 24.6 | 19.5 | 29.7 | 23 | 7.56 | 6.53 | 331 | 6.28 | 6.32 | 1.21 | 0.98 | -5.67 |
| ComENet† | 45 | 32.4 | 23.1 | 19.8 | 24.5 | 22 | 7.98 | 6.86 | 259 | 6.82 | 6.69 | 1.20 | 0.93 | -5.69 |
| SphereNet† | 46 | 31.1 | 22.8 | 18.9 | 24.5 | 22 | 7.78 | 6.33 | 268 | 6.36 | 6.26 | 1.12 | 0.91 | -5.73 |
| Scalarization-based models | ||||||||||||||
| ClofNet | 63 | 53 | 33 | 25 | 40 | 27 | 9 | 9 | 610 | 9 | 8 | 1.23 | 1.37 | -5.37 |
| EGNN | 71 | 48 | 29 | 25 | 29 | 31 | 12 | 12 | 106 | 12 | 11 | 1.55 | 1.23 | -5.43 |
| PaiNN† | 45 | 45.7 | 27.6 | 20.4 | 12.0 | 24 | 7.35 | 5.98 | 66 | 5.83 | 5.85 | 1.28 | 1.01 | -5.85 |
| LEFTNet | 48 | 40 | 24 | 18 | 12 | 23 | 7 | 6 | 109 | 7 | 6 | 1.33 | 0.91 | -5.82 |
| EQGAT | 53 | 32 | 20 | 16 | 11 | 24 | 23 | 24 | 382 | 25 | 25 | 2.00 | 0.86 | -5.28 |
| ET | 59 | 36.1 | 20.3 | 17.5 | 11 | 26 | 7.62 | 6.16 | 33 | 6.38 | 6.15 | 1.84 | 0.84 | -5.90 |
| Geoformer | 40 | 33.8 | 18.4 | 15.4 | 10 | 22 | 6.13 | 4.39 | 28 | 4.41 | 4.43 | 1.28 | 0.75 | -6.12 |
| SaVeNet-B† | 39 | 24.8 | 18.4 | 16.3 | 9.3 | 23 | 6.64 | 5.43 | 58 | 5.48 | 5.43 | 1.18 | 0.69 | -6.04 |
| High-degree steerable models | ||||||||||||||
| SEGNN | 60 | 42 | 24 | 21 | 23 | 31 | 15 | 16 | 660 | 13 | 15 | 1.62 | 1.08 | -5.27 |
| HDGNN† | 46 | 32 | 18 | 16 | 17 | 23 | 11 | 10 | 342 | 8.12 | 8.34 | 1.21 | 0.80 | -5.64 |
| Equiformer | 46 | 30 | 15.4 | 14.7 | 12 | 23 | 7.63 | 6.63 | 251 | 6.74 | 6.59 | 1.26 | 0.70 | -5.82 |
| EquiformerV2 | 47 | 29.0 | 14.4 | 13.3 | 9.9 | 23 | 7.57 | 6.22 | 186 | 6.49 | 6.17 | 1.47 | 0.67 | -5.87 |
| Pre-trained models | ||||||||||||||
| Transformer-M | 41 | 27.4 | 17.5 | 16.2 | 37 | 22 | 9.63 | 9.39 | 75 | 9.41 | 9.37 | 1.18 | 0.86 | -5.74 |
| SE(3)-DDM | 46 | 40.2 | 23.5 | 19.5 | 15 | 24 | 7.65 | 7.09 | 122 | 6.99 | 6.92 | 1.31 | 0.93 | -5.76 |
| 3D-EMGP | 57 | 37.1 | 21.3 | 18.2 | 20 | 26 | 9.30 | 8.70 | 92 | 8.60 | 8.60 | 1.38 | 0.92 | -5.68 |
| Coord | 52 | 31.8 | 17.7 | 14.3 | 12 | 20 | 6.91 | 6.45 | 450 | 6.11 | 6.57 | 1.71 | 0.76 | -5.75 |
| Frad(VRN) | 42 | 27.7 | 17.9 | 13.8 | 11 | 21 | 6.03 | 6.01 | 354 | 5.35 | 5.41 | 1.63 | 0.71 | -5.85 |
| Frad(RN) | 37 | 27.8 | 15.3 | 13.7 | 10 | 20 | 6.19 | 5.55 | 342 | 5.62 | 5.33 | 1.42 | 0.66 | -5.91 |
| Spherical-scalarization models | ||||||||||||||
| GotenNet S | 34 | 23.2 | 16.3 | 14.7 | 7.5 | 20 | 5.51 | 3.86 | 27 | 3.76 | 3.82 | 1.15 | 0.62 | -6.27 |
| GotenNet B | 32 | 21.3 | 15.2 | 13.5 | 7.3 | 20 | 5.33 | 3.52 | 24 | 3.49 | 3.49 | 1.10 | 0.58 | -6.33 |
| GotenNet L | 28 | 19.9 | 13.7 | 12.2 | 7.7 | 19 | 5.05 | 3.33 | 22 | 3.41 | 3.37 | 1.08 | 0.53 | -6.39 |
Bold underlined cells are the lowest value in their column, computed by this page from the printed cells; ties are all marked. † marks rows the paper says use different data partitions. Units are the paper’s own Units row.
Molecule3D: random and scaffold splits, Table 2
Measured, Table 2The paper describes Molecule3D as containing “more than 3 million molecules”. Table 2 names neither a metric nor units, so the values are called errors as printed. Lower is better.
The full table: 11 rows as printed in Table 2
| Model | μ | εHOMO | εLUMO | Δε | std. | log | Δε, scaffold split |
|---|---|---|---|---|---|---|---|
| GIN-Virtual | .0882 | .0692 | .0632 | .1036 | .0592 | -2.87 | .2371 |
| SchNet | .0532 | .0275 | .0265 | .0428 | .0263 | -3.66 | .1511 |
| DimeNet++ | .0293 | .0240 | .0190 | .0306 | .0188 | -4.01 | .1214 |
| SphereNet | .0288 | .0239 | .0183 | .0301 | .0184 | -4.03 | .1182 |
| ComENet | .0345 | .0288 | .0252 | .0326 | .0220 | -3.84 | .1273 |
| PaiNN | .0196 | .0263 | .0197 | .0307 | .0182 | -4.08 | .1208 |
| ET | .0223 | .0199 | .0194 | .0303 | .0170 | -4.13 | .1282 |
| LEFTNet | .0151 | .0183 | .0157 | .0275 | .0145 | -4.32 | .1317 |
| SaVeNet-L | .0136 | .0159 | .0143 | .0239 | .0128 | -4.44 | .1082 |
| Geoformer | not reported | not reported | not reported | .0202 | not reported | not reported | .1135 |
| GotenNet | .0103 | .0108 | .0112 | .0165 | .0103 | -4.65 | .1002 |
Bold underlined cells are the lowest value in their column, computed from the printed cells. “Not reported” stands for a dash in the paper.
MD22: seven systems, 42 to 370 atoms, Table 3
Measured, Table 3Molecular dynamics trajectories for seven systems. Mean absolute error of energy in kcal/mol and of forces in kcal/mol/Å, lower is better. The small number beside each name is the node count.
The full table: 14 rows as printed in Table 3
| Molecule | Nodes | E or F | sGDML | ET | Allegro | MACE | Equiformer | ViSNet | QuinNet | E-LSRM | V-LSRM | SO3KRATES | GotenNet S | GotenNet B |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Tetrapeptide | 42 | E | 0.3902 | 0.1121 | 0.1019 | 0.0620 | 0.0828 | 0.0796 | 0.0840 | 0.0780 | 0.0654 | 0.337 | 0.0589 | 0.0505 |
| Tetrapeptide | 42 | F | 0.7968 | 0.1879 | 0.1068 | 0.0876 | 0.0804 | 0.0972 | 0.0681 | 0.0887 | 0.0902 | 0.244 | 0.0719 | 0.0567 |
| DHA | 56 | E | 1.3117 | 0.1205 | 0.1153 | 0.1317 | 0.1788 | 0.1526 | 0.1200 | 0.0878 | 0.0873 | 0.379 | 0.0640 | 0.0575 |
| DHA | 56 | F | 0.7474 | 0.1209 | 0.0732 | 0.0646 | 0.0506 | 0.0668 | 0.0515 | 0.0534 | 0.0598 | 0.242 | 0.0496 | 0.0421 |
| Stachyose | 87 | E | 4.0497 | 0.1393 | 0.2485 | 0.1244 | 0.1404 | 0.1283 | 0.2300 | 0.1252 | 0.1055 | 0.442 | 0.0751 | 0.0673 |
| Stachyose | 87 | F | 0.6744 | 0.1921 | 0.0971 | 0.0876 | 0.0635 | 0.0869 | 0.0543 | 0.0632 | 0.0767 | 0.435 | 0.0512 | 0.0427 |
| AT-AT | 60 | E | 0.7235 | 0.1120 | 0.1428 | 0.1093 | 0.1309 | 0.1688 | 0.1400 | 0.1007 | 0.0772 | 0.178 | 0.0640 | 0.0544 |
| AT-AT | 60 | F | 0.6911 | 0.2036 | 0.0952 | 0.0992 | 0.0960 | 0.1070 | 0.0687 | 0.0881 | 0.0781 | 0.216 | 0.0632 | 0.0478 |
| AT-AT-CG-CG | 118 | E | 1.3885 | 0.2072 | 0.3933 | 0.1578 | 0.1510 | 0.1995 | 0.3800 | 0.1335 | 0.1135 | 0.345 | 0.0964 | 0.0923 |
| AT-AT-CG-CG | 118 | F | 0.7028 | 0.3259 | 0.1280 | 0.1153 | 0.1252 | 0.1563 | 0.1273 | 0.1065 | 0.1063 | 0.332 | 0.0824 | 0.0744 |
| Buckyball catcher | 148 | E | 1.1962 | 0.5188 | 0.5258 | not reported | 0.3978 | 0.4421 | 0.5624 | not reported | 0.4220 | 0.381 | 0.3432 | 0.3032 |
| Buckyball catcher | 148 | F | 0.6820 | 0.3318 | 0.0887 | not reported | 0.1114 | 0.1335 | 0.1091 | not reported | 0.1026 | 0.237 | 0.0838 | 0.0789 |
| Double-walled nanotube | 370 | E | 4.0122 | 1.4732 | 2.2097 | not reported | 1.1945 | 1.0339 | 1.8130 | not reported | 1.8230 | 0.993 | 0.9993 | 0.6641 |
| Double-walled nanotube | 370 | F | 0.5231 | 1.0031 | 0.3428 | not reported | 0.2747 | 0.3959 | 0.2473 | not reported | 0.3391 | 0.727 | 0.2464 | 0.1888 |
Bold underlined cells are the lowest value in their row, computed from the printed cells. “Not reported” stands for a dash in the paper. Molecules are in the table’s printed order.
rMD17: ten small molecules, Table 4
Measured, Table 4Ten small organic molecules with 100,000 conformations each; 950 for training, 50 for validation, the rest for testing, averaged over five predefined splits. Mean absolute error of energy in kcal/mol and of forces in kcal/mol/Å.
The full table: 20 rows as printed in Table 4
| Molecule | E or F | NequIP | ACE | UNiTE | Allegro | BOTNet | MACE | TensorNet | GotenNet |
|---|---|---|---|---|---|---|---|---|---|
| Aspirin | E | 0.0530 | 0.1407 | 0.0553 | 0.0530 | 0.0530 | 0.0507 | 0.0553 | 0.0358 |
| Aspirin | F | 0.1891 | 0.4128 | 0.1753 | 0.1683 | 0.1960 | 0.1522 | 0.2052 | 0.1306 |
| Azobenzene | E | 0.0161 | 0.0830 | 0.0254 | 0.0277 | 0.0161 | 0.0277 | 0.0161 | 0.0121 |
| Azobenzene | F | 0.0669 | 0.2514 | 0.0969 | 0.0600 | 0.0761 | 0.0692 | 0.0715 | 0.0483 |
| Benzene | E | 0.0009 | 0.0009 | 0.0016 | 0.0069 | 0.0007 | 0.0092 | 0.0005 | 0.0005 |
| Benzene | F | 0.0069 | 0.0115 | 0.0168 | 0.0046 | 0.0069 | 0.0069 | 0.0069 | 0.0047 |
| Ethanol | E | 0.0092 | 0.0277 | 0.0143 | 0.0092 | 0.0092 | 0.0092 | 0.0115 | 0.0071 |
| Ethanol | F | 0.0646 | 0.1683 | 0.0853 | 0.0484 | 0.0738 | 0.0484 | 0.0807 | 0.0482 |
| Malonaldehyde | E | 0.0184 | 0.0392 | 0.0254 | 0.0138 | 0.0184 | 0.0184 | 0.0184 | 0.0129 |
| Malonaldehyde | F | 0.1176 | 0.2560 | 0.1522 | 0.0830 | 0.1338 | 0.0945 | 0.1245 | 0.0830 |
| Naphthalene | E | 0.0208 | 0.0208 | 0.0106 | 0.0046 | 0.0046 | 0.0115 | 0.0046 | 0.0039 |
| Naphthalene | F | 0.0300 | 0.1176 | 0.0600 | 0.0208 | 0.0415 | 0.0369 | 0.0369 | 0.0240 |
| Paracetamol | E | 0.0323 | 0.0922 | 0.0438 | 0.0346 | 0.0300 | 0.0300 | 0.0300 | 0.0212 |
| Paracetamol | F | 0.1361 | 0.2929 | 0.1637 | 0.1130 | 0.1338 | 0.1107 | 0.1361 | 0.0928 |
| Salicylic acid | E | 0.0161 | 0.0415 | 0.0168 | 0.0208 | 0.0184 | 0.0208 | 0.0184 | 0.0141 |
| Salicylic acid | F | 0.0922 | 0.2145 | 0.0876 | 0.0669 | 0.0992 | 0.0715 | 0.1061 | 0.0703 |
| Toluene | E | 0.0069 | 0.0254 | 0.0104 | 0.0092 | 0.0092 | 0.0115 | 0.0069 | 0.0044 |
| Toluene | F | 0.0369 | 0.1499 | 0.0577 | 0.0415 | 0.0438 | 0.0346 | 0.0392 | 0.0261 |
| Uracil | E | 0.0092 | 0.0254 | 0.0134 | 0.0138 | 0.0092 | 0.0115 | 0.0092 | 0.0064 |
| Uracil | F | 0.0715 | 0.1522 | 0.0876 | 0.0415 | 0.0738 | 0.0484 | 0.0715 | 0.0417 |
Bold underlined cells are the lowest value in their row, computed from the printed cells; ties are all marked. The paper writes that GotenNet leads in 80% of tasks. That is 16 of 20 rows, counting two ties.
05 What it costs, and which part matters
The timed variants have lower latency than the three attention-based baselines the paper timed. In the ablation, the part that matters most is the geometric encoding.
The paper measures cost in one table and takes the design apart in another. Both come with conditions, and they are printed next to the charts, not under a tooltip.
Figure 6Measured, Table 8
Choose a cost and an accuracy, both from the same table
Read this with the chart
- These are the shared-coefficient variants. The paper writes them with a hat over the letter; here they are S-hat, B-hat and L-hat. Their std. MAE is 0.67, 0.61 and 0.56, not the 0.62, 0.58 and 0.53 of Table 1. The paper gives no timing for the headline variants.
- Latency is measured at a batch size of 128 for every model. The batch size column is the training batch size behind the epoch time and the GPU days.
- Training protocols differ: 300 epochs for Equiformer and EquiformerV2, up to 600 with early stopping for Geoformer. GotenNet’s QM9 configuration in Table 6 lists 1,000 epochs.
- Not everything favours GotenNet. Equiformer has the fewest parameters (3.5M), and its 1.48 GPU days is below the largest timed variant’s 1.87 average.
- The paper does not name the dataset these timings were taken on. The accuracy columns equal its QM9 values.
GPU days as printed: minimum to maximum, average, and limit
The bar runs from the minimum to the maximum, the tick is the average, the hollow triangle the limit. EquiformerV2’s limit is printed as 2.65, below its other three values, and is shown as printed. For Geoformer the paper prints only a limit.
Table 8 as printed
| Model | Batch size | Time per epoch | GPU days, min | GPU days, avg | GPU days, max | GPU days, limit | Training latency | Inference latency | Trainable parameters | Std. MAE | Log MAE |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Equiformer | 128 | 425 | 1.48 | 1.48 | 1.48 | 1.48 | 421 | 150 | 3.5M | 0.70 | -5.82 |
| EquiformerV2 | 64 | 821 | 2.85 | 2.85 | 2.85 | 2.65 | 918 | 341 | 11.2M | 0.67 | -5.87 |
| EquiformerV2 | 48 | 847 | 2.94 | 2.94 | 2.94 | 2.65 | 918 | 341 | 11.2M | 0.67 | -5.87 |
| Geoformer | 32 | 436 | not reported | not reported | not reported | 5.05 | 759 | 264 | 50.6M | 0.75 | -6.12 |
| GotenNet S-hat | 32 | 117 | 0.41 | 0.75 | 1.34 | 1.35 | 80 | 37 | 6.1M | 0.67 | -6.21 |
| GotenNet B-hat | 32 | 180 | 0.75 | 1.15 | 1.92 | 2.08 | 120 | 56 | 9.2M | 0.61 | -6.26 |
| GotenNet L-hat | 32 | 291 | 1.37 | 1.87 | 2.33 | 3.37 | 244 | 112 | 18.3M | 0.56 | -6.34 |
Figure 7Measured, Table 5
Load one of the eight rows the paper actually ran
| Row | # L | Lmax | SE | SEA | GE | HTR | std | log |
|---|---|---|---|---|---|---|---|---|
| 4 layers | 4 | 2 | on | on | on | on | 0.67 | -6.21 |
| L max = 1 | 6 | 1 | on | on | on | on | 0.68 | -6.17 |
| Without structural embedding | 6 | 2 | off | on | on | on | 0.67 | -6.17 |
| Without self-attention | 6 | 2 | on | off | on | on | 0.65 | -6.23 |
| Without geometric encoding | 6 | 2 | on | on | off | on | 0.83 | -5.96 |
| Without HTR | 6 | 2 | on | on | on | off | 0.64 | -6.20 |
| Full model, 6 layers | 6 | 2 | on | on | on | on | 0.61 | -6.26 |
| 12 layers | 12 | 2 | on | on | on | on | 0.56 | -6.34 |
There are no free switches because only these eight combinations were run. The ablation is on QM9 only, with no seeds, repeated runs or error bars reported, and no row removes the feed-forward block. The three rows with every component on print the same std and log as the shared-coefficient variants of Tables 7 and 8; the paper does not say which variant the ablation used.
06 Office hours
The night shift answers.
Written by Tian's agent while Tian slept and Pepper supervised. Pepper accepts no liability either.
Each answer says where in the paper it comes from, so you can check the agent's work.
It says no irreps and no Clebsch-Gordan transforms, but the features are spherical harmonics of degree 1 and 2. What was actually removed?
The features are still degree-l objects, initialised from spherical harmonics up to Lmax = 2 and computed with the e3nn library, and the appendix proof rotates them with Wigner D-matrices. What the equations leave out is any step that multiplies two steerable features into a new steerable feature. Steerable features are scaled by invariant coefficients and summed, gated by an invariant, and meet each other only inside an inner product whose result is an invariant edge update.
The paper’s own label for this is “spherical-scalarization”, and it credits the inner-product idea to earlier work.
Source: Section 3.1, Equations 8 and 10 to 13, Appendix B; Section 1, page 2How much cheaper is it, and is the comparison like for like?
The paper gives measured latency, not an operation count. In Table 8, at a batch size of 128, training and inference latency are 80 and 37 ms, 120 and 56 ms, and 244 and 112 ms for the three timed GotenNet variants, against 421 and 150 ms for Equiformer, 918 and 341 ms for EquiformerV2, and 759 and 264 ms for Geoformer.
The conditions: the timed variants are the shared-coefficient ones, whose QM9 std. MAE is higher, that is worse (0.67, 0.61, 0.56 against 0.62, 0.58, 0.53), and the headline models were not timed. Equiformer has fewer parameters than any GotenNet row, and its 1.48 GPU days is below the largest timed variant’s 1.87 average. Training protocols differ, and only three baselines were timed. The paper gives no complexity class, operation count or memory measurement for GotenNet, and lists a formal analysis of the trade-off as future work.
Source: Table 8 and Appendix H, page 22; Appendix L, page 25Does it win everywhere?
No, and the tables say where. On QM9 the S and B sizes are not below every baseline on εHOMO and εLUMO: EquiformerV2 prints 14.4 and 13.3 against B’s 15.2 and 13.5 and S’s 16.3 and 14.7. S is also not below SphereNet on ZPVE (1.12 against 1.15), a row that used a different data partition. The L size prints a lower value than every other model’s row in every column of Table 9, although on μ the B size (7.3) and the S size (7.5) print lower values than L (7.7).
On MD22 the B size prints the lowest error in all 14 rows. The S size is lower than every baseline in 12 of the 14; the exceptions are Tetrapeptide forces (QuinNet 0.0681 against 0.0719) and Double-walled nanotube energy (SO3KRATES 0.993 against 0.9993). On rMD17 Allegro prints lower force errors on four molecules and two more rows are ties. Apart from rMD17’s five predefined splits, the paper reports no seeds, error bars or significance tests.
Source: Tables 1, 3, 4 and 9, computed from the printed cellsWhich part of the design is doing the work?
In the QM9 ablation at six layers, removing the geometric encoding moves std. MAE from 0.61 to 0.83, the largest change. Lmax = 1 gives 0.68, no structural embedding 0.67, no self-attention 0.65, no HTR 0.64. Depth matters too: 0.67, 0.61 and 0.56 at 4, 6 and 12 layers.
Removing the inner-product refinement, the component this page spends the most time on, gives the smallest change in std. MAE in that table. On log MAE, removing self-attention changes less (−6.23, against −6.20 without HTR; the full model prints −6.26). This ranking is read from the Table 5 cells. The paper itself singles out only the geometric encoding and Lmax. The ablation is on QM9 only, reports no seeds, repeated runs or error bars, and has no row that removes the feed-forward block.
Source: Table 5 and Section 4.5, page 10What was not tested?
Degrees above 2: Lmax is 2 in every reported benchmark result, the only other value tried is 1, in one ablation row, and higher degrees are listed as future work. Systems larger than 370 atoms; the timings cover 10 to 140 nodes. The paper reports no molecular dynamics simulation: its force-field results are energy and force errors on test conformations. Proteins, materials and point clouds are named as possible extensions only.
Equivariance is proved in the appendices, not measured numerically. The paper has no limitations section; these limits are read from what the experiments cover.
Source: Appendix E, page 21; Table 3, page 9; Section 4.2, page 9; Appendix L, pages 24 to 25; Appendices A to C