All this might have changed with the introduction of Agentic AI, which significantly decreased the effort and barrier for coding, running experiments, analysing, and writing scientific papers. More surprisingly, it has also lowered the entry barrier on the required technical and scientific skills and understanding of targeted topics. For work in deep learning and computer vison, for instance, Claude has now more than reasonable understanding of how neural network hyper parameters impact training and it baffles me how well it can analyze experiments and suggest changes to code and settings.

The reaction to the impact of AI has been divided in many areas, and largely questioned in areas like art. I think that science is still in the process of digesting the impact of Agentic AI and in this short post I definitely do not want to question AI usage. I actually think that its long term impact will be positive and disruptive.

Rather, I would like to discuss its impact on science communication and how knowledge is dissiminated. There is a now a real danger of the community being overrun by papers which Agentic AI made much cheaper to produce.

This raises the question of the intrinsic value of a scientific paper: evaluating a paper on the merits of novelty alone ignores the human cost of actually reading papers. Unless we switch to a mode where papers are written by AI agents to be read by AI agents alone, the human costs of reading papers need to be factored in. If the number of papers with sufficient “novelty” increases 20 fold but the capacity for reading papers stays constant, something needs to give, and that is most likely the threshold for sufficient novelty.

This is not necessarily surprising, for two reasons: because of necessity (we can’t read the sheer amount of papers written — we couldn’t in the past and this will worsen signficantly), but also because of opportunity cost. The value of reading a (relatively cheaply produced) paper is lower if my own effort for producing similar work has been lowered by Agentic AI, in particular if my own work and experiments will be closer to my target setting than the work of the paper I would read. In the past it was cheaper to read a paper somewhat related to my work than doing the actual work myself. This comparison might have shifted for “coding bound” research and in fine Agent AI will decrease the value of “coding bound” science and increase the value of “data bound” or “compute bound” science, ie. work whose total cost is not determined by implementations and writing but by data collection, data possession and by compute. It will also increase the value of experimental research not available to agents: experiments with real (!) robots, biological reactors etc.

P.S.

(No LLM has been used in writing this text and em dashes in this text have been placed by the human author)

P.P.S.

Usage of synthethised music in the 1970ies and 1980ies: the covers of the British Rock band “Queen” in 1976 (top) and in 1980 (bottom)

From “Kinaema”, our robotics sequences model (https://arxiv.org/abs/2510.20261, NeurIPS 2025):

From “What really matters in ImageGoal navigation”, to appear in 3DV 2025 (https://arxiv.org/abs/2507.01667):

From “Zero-BEV”, published in 3DV 2024 (https://arxiv.org/abs/2402.13848)

A couple of years ago I started drawing figures like these with Google Sketchup, a really easy to use software with a shallow learning curve. But first, I quickly reached its limits, and second, Google sold it and it stopped to be free. I switched to Blender, which has a reputation of being very difficult to use with a steep learning curve … But I found out that this only applies if you really need the very advanced features, and if you have a hateful relationship with tutorials. If you are ready to spend around 2h to watch a series of very short tutorial videos like this excellent series here by “Blender Guru”’, you are quickly set up to do incredible figures.

Some specific tipps for scientific Figures

Most Figures in scientifc papers are simple shapes, very often just simple planes, but they are textured. It is actually not straightforward to add textures to planes, but it is very easy to just transform image files (.jpg, .gif, .png) into … planes, with the “Images as Planes” add-on. Once activated, load an image:

In the file open dialog, be sure that the “Emit” property is set to a reasonable value so that the created object emits some light and is nicely visible:

Don’t forget to switch to viewport shading so that textures are correctly displayed:

You should then see something like this, where in the properties editor on the right of the window you can edit size, rotation, position etc. Alternatively, to freely move it around with the mouse, select the object, hit “g” and drag it around.

To make a simple video sequence like Figure, I just duplicate the frame with a right click “Duplicate objects”, hit the enter key immediately (don’t move the mouse) to keep it its current position, then edit the y-axis coordinate in the properties panel to shift it 0.3m, rinse and repeat:

I then set my active camera to the current view:

Rendering this with F12 (you need to play around to get the optimal view), is underwhelming: the background is not white, and we don’t have “edges”, ie. no black frames around the images, which are really useful in the kind of diagrams we want to see in papers:

Edges are easy: in the camera panel (camera icon), check pane/box “Freestyle” and you can optionally modify line thickness.

In order to get a transparent background, do the following. In the “Editor Type” Panel, pane “Film”, be sure to check “Transparent”:

After rendering, when you save your image, be sure to save it in PNG format (JPGs do not handle transparency) and switch the color format to “RGBA”, where “A” stands for “alpha channel”, which encodes transparency:

And voilà:

A different viewpoint:

I then normally load this into Powerpoint and add the 2D parts of the Figure, like arrows and boxes:

Files for this tutorial: [Blender file] [Powerpoint file]

The content of this post in a nutshell:

Capacity and representation size in recurrent neural networks

In their most classical version, recurrent neural networks keep a recurrent memory h, which is updated at each time instant given an observation x and two parameter matrices U and W:

More modern variants like LSTMs or GRUs differ in that additional gates and non-linearities are added, but they do not change the essence of what will be discussed here. As can be seen in a graphical version of the equation below, RNNs tightly couple two different properties of the model, which we would like to adjust separately: the size of the hidden state h, and the capacity of the network:

In particular, the size of the transition weight matrix U is a square of the dimension of the state h. For vanilla RNNs, the capacity of the model therefore can’t be decreased without decreasing state dimensions, although we sometimes would like to decouple these two properties: the dimension of the state governs the amount of information we need to propagate over time, whereas the capacity governs the complexity of the mathematical function approximating the dynamic of the system. There is no reason why these two properties should be related.

Several solutions have been proposed for this problem:

(i) low rank decompositions of the transition matrix U, or similar mathematical constraints put on its structure, for instance [2].

(ii) Remove a part of the hidden state and transfer it into an “external” representation, which is not replaced in its entirety at each iteration, but rather updated sparsely through attention, which decreases the capacity necessary for updates. This is the strategy employed by Memory Networks [3], Neural Turing Machines [4] and their variants and follow-ups.

The two solutions share similar goals but achieve them differently. The first approach (i) keeps the mathematical structure of recurrent networks and adds constraints on one of the weight matrices, which is a smaller change to the method. In contrast, (ii) fundamentally changes how the model is structured by outsourcing memory to an external component.

Transformers and complexity

Transformers [5] are neural networks built on query, key, value attention using dot-product similarity. Without going into details of how transformers work, the expensive part is the calculation of pairwise similarities between items of a set

or, expressed as a softmax over an attention matrix A,

where X is the set of items, each item represented as vector, and W and V are two weight matrices, which project linearly project the input vectors into a space in which the dot-product similarity is meaningful (in the sense of the task to learn). It is easy to see that this operation is O(N²), which creates troubles in applications where the number of items is big, like images [6][7], videos [8] etc. One strategy is to sparsify interactions and allow each layer to attend to a subset of items only [8], which solves the problem externally and does not touch on the concept of transformers itself.

Two of the most popular attempts to address this problem bear some resemblance to the two different ways (i) and (ii) for decoupling capacity and memory representation in RNNs. However, instead of focusing on presentation size, here the focus is on the calculations and interactions between data items:

a) instead of performing the full attention calculations for all item pairs, involving a full attention matrix A, the matrix can be decomposed using a matrix low rank decomposition, which can lead to a more efficient computation [10, 11] and others. The underlying hypothesis is that the attention operation itself is mostly low rank and can be described by a couple of singular values. This is somewhat similar to decomposing the RNN update equation, and thus enforcing a low rank of the dynamic of a dynamic system.

b) A recent paper [12] proposes an alternative approach, somewhat similar to (ii), which externalizes the problem. Instead of allowing all possible pairwise interactions of input items, the interactions are transferred into an external (latent) representation. The “Perceiver” [12], see figure below, allows self-attention only in this low dimensional latent space where attention operations are cheaper. The high-dimensional input space only interacts with the latent representation, never with itself (no self-attention in input space).

The comparison might be a stretch, but it is still interesting. As in the first problem (decoupling capacity and representation size in RNNs), the two solutions (a) and (b) share similar goals but achieve them differently. The first approach (a) keeps the mathematical structure of transformers and adds constraints on the shape of the attention matrix, which is a smaller change to the method. In contrast, (b) fundamentally changes how the model is structured by outsourcing interactions to an external component.

Twitter: @chriswolfvision

References

[1] Natalia Neverova, Christian Wolf, Griffin Lacey, Lex Fridman, Deepak Chandra, Brandon Barbello and Graham W. Taylor. Learning Human Identity from Motion Patterns. IEEE Access (4):1810–1820, 2016.

[2] Cijo Jose, Moustapha Cisse, Francois Fleuret, Kronecker Recurrent Units, ICML 2018.

[3] Jason Weston, Sumit Chopra, Antoine Bordes, Memory Networks, ICLR 2015.

[4] Alex Graves, Greg Wayne, Ivo Danihelka, Neural Turing Machines, arXiv:1410.5401, 2014.

[5] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, Attention Is All You Need, NeurIPS 2017.

[6] Xiaolong Wang, Ross Girshick, Abhinav Gupta, Kaiming He, Non-local Neural Networks, CVPR 2018.

[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, ICLR 2021.

[8] Brendan Duke, Abdalla Ahmed, Christian Wolf, Parham Aarabi and Graham W. Taylor, SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentation, CVPR 2021.

[9] Shuangfei Zhai, Walter Talbott, Nitish Srivastava, Chen Huang, Hanlin Goh, Joshua M. Susskind, An Attention Free Transformer, openreview 2020 (ICLR 2020 reject).

[10] Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, Vikas Singh, Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention, arXiv:2102.03902, 2021.

[11] Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, Hao Ma, Linformer: Self-Attention with Linear Complexity, arxiv:2006.04768, 2020.

[12] Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, Joao Carreira, Perceiver: General Perception with Iterative Attention, arXiv:2103.03206, 2021.

The context to this post are recent papers providing evidence that prediction performance of deep networks trained for image classification is in majority tue to texture biases, i.e. networks take decisions based on textures / small patches :

• Geirhos et al., ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness, ICLR 2019.
• Hermann et la., The Origins and Prevalence of Texture Bias in Convolutional Neural Networks, NeurIPS 2020.
• ???, Shape or Texture: Understanding Discriminative Features in CNNs, submitted to ICLR 2021.
• Brendel et al., Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet, ICLR 2019. (“BagNet”).

I then stumbled across this nice image on Twitter, shared by https://twitter.com/SolTight

What follows, is a summary of a Twitter thread I started.

I passed this image through a pretrained ResNet 101 with PyTorch, and at first sight the results seem to be counterintuitive? It predicts:

Tiger shark 23%
Hammerhead 21%
Great white shark 16%
Gar, garfish 11%
Sturgeon 3%

So … at first sight, shape > [ texture + mountain context ], at least this seems to be the case for this example. Note that we do not really know what the answer should be … the species does not exist. But for ResNet-101, the shape of the object seems to be more important than color, texture, and the fact that the background is in the mountains and not under water.

A nice baseline is to pass the the same image through BagNet, which, roughly divides the image into small patches, creates a representation by patch, and then puts all these representations into a “bag”, i.e. a structure which does not handle spatial relationships between the different patches. This kind of neural network has a strong inductive bias towards texture, as shape requires spatial relationships, at least if the scale of the shape is bigger than the patches of the decomposition.

So, BagNet provides the following results:

Sorrel 81%
Ibizan hound, Ibizan Podenco 11%
Basenji 4%
Gazelle 2%
Ox 1%

This is not surprising, the prediction is now clearly biased towards textures, which is what BagNet actually sees.

But if we investigate this a bit further, and check the results of the originally used ResNet-101 model, we can find that our intuition (shape > texture) might be wrong also for this example. Let’s check the results for two different cropped images:

ResNet-101
Hammerhead 52%
Great white shark 25%
Tiger shark 16%
Killer whale 6%
Sea lion 0.1%

BagNet-33
Electric ray 10%
Sturgeon 9%
Sorrel 7%
Tiger shark 7%
Stingray 5%

And:

ResNet-101
Sorrel 92%
Ox 6%
Rhodesian ridgeback 0.4%
Ibizan hound 0.2%
Boxer 0.2%

BagNet-33
Sorrel 24%
Gyromitra 16%
Ox 6%
Hog, pig 4%
Flute, transverse flute 4%

Clearly, cropping more and more information from the image brings the prediction of the more powerful ResNet-101 closer to the texture biased BagNet. In particular, this fin on the back of the creatures seems to be a killer feature.

A sanity check of the model output on a cow :

ResNet-101
Ox 93%
Oxcart 3%
Gazelle 0.8%
Water buffalo, water ox 0.8%
Ram, tup 0.7%

BagNet-33
ox 98%
oxcart 0.5%
Arabian camel, dromedary, Camelus dromedarius 0.3%
water buffalo, water ox, Asiatic buffalo, Bubalus bubalis 0.1%
bison 0.1%

And finally, one more variant of the original image:

ResNet-101
Hammerhead 53%
Great white shark 14%
Triceratops 8%
Airship 6%
Tiger shark 3%

BagNet-33
Albatross, mollymawk 35%
Goose 15%
Killer whale 7%
Drake 6%
Whippet 4%

First, we see that the amount of parameters for the first trainable layer is extremely high, since each of the 480 000 input values connected to each of the 500 hidden units, requiring 480 000 parameters (not counting bias values). Current state-of-the-art computer vision models have around 20–30 million trainable parameters, but bigger models exist. More importantly, these parameters are usually distributed over a much larger number of layers, not a single layer.

Secondly, what will happen if the object of interest (here, the monkey, let us suppose we would like to recognize monkeys in images) changes its position in the image?

The fact that the monkey moved from the left part of the image to the right part of the image means that different pixels of the image contain the monkey, which again means that different trainable parameters are responsable for firing when the monkey appears. As a consequence, the same reasoning (“fire when monkey appears”) needs to be performed, and thus learned, for different subsets of parameters corresponding to different locations. This is a tremendous waste of computation, representation, and requires the network to relearn the same reasoning, again and again, multiple times. This will also hurt generalization performance of the model.

Our real objective is to find a neural mapping from images to predictions, which is invariant to all different geometric transformations to which objects could be subject to, for instance translations, rotations and projective distortions. For humans and animals, who are articulated “objects”, this would also include quite complex deformations. Let us focus on translations only for the moment.

Translation invariance requires, that the output of a mapping / network does not change when the input is translated. This is possible to achieve approximatively, up to a certain amount, but not directly.

We will first investigate a related property, which is translation equivariance.

What is an equivariant mapping?

In short, an equivariant mapping is a mapping which preserves the algebric structure of a transformation. As a particular case, a translation equivariant mapping is a mapping which, when the input is translated, leads to a translated mapping, as illustrated in the following animation:

When the input image at the left (a digit taken from the MNIST dataset and embedded in a larger canvas) is translated by a certain amount, the output feature map is translated by the same amount.

Formally this property is illustrated by the figure below:

Input image X1, showing digit “4” is translated to right, which gives input image X2. F1 and F2 are respectively, the feature maps calculated by a translation equivariant mapping ϕ. In this case, the feature map F2, obtained by passing X2 through ϕ is equivalent to feature map obtained by applying to feature map F1 the same translation T which also had been applied to X1 to get X2.

The animation above has been created by training a 4-layer convolutional neural network on the original MNIST dataset of digits of size 28x28 pixels. The network has the following architecture:

Each layer has kernel size 5x5. The convolutional block is followed by a single fully connected layer, the output layer. Here is the PyTorch code of the model:

class LeNet(torch.nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = torch.nn.Conv2d(1, 20, 5, 1)
        self.conv2 = torch.nn.Conv2d(20, 40, 5, 1)
        self.conv3 = torch.nn.Conv2d(40, 40, 5, 1)
        self.conv4 = torch.nn.Conv2d(40, 5, 5, 1)
        self.fc1 = torch.nn.Linear(6*6*5, 10)

def forward(self, x):
        x = F.relu(self.conv1(x)) # now 24x24        
        x = F.relu(self.conv2(x)) # now 20x20
        x = F.relu(self.conv3(x)) # now 16x16
        x = F.relu(self.conv4(x)) # now 12x12
        x = F.max_pool2d(x, 2, 2) # now 6x6        
        x = x.view(-1, 6*6*5)
        return self.fc1(x)

Training: The model was first trained with cross-entropy loss until convergence. It is not the best model for MNIST, and it achieved a validation error of ~1%. In particular, compared to more frequent models it does not perform any pooling between the convolutional layers, which we here did on purpose.

The feature map shown in the animation above has been produced by the 4th convolutional layer, just before the pooling layer. For the illustration, we performed a k-means clustering of the scalar activations on a batch of 50 images and assigned a color code to each cluster.

Testing/animation: while the model was trained on images of size 28x28, to produce the animations we fed images of larger size (38x38) into the network in order to be able move digits over the input image. It is interesting to note, that this does not require a change of the convolutional part of the model, as convolutions are independent of the input size of the image. It would, however, require a change of the fully connected output layer, which was not used for the visualization above.

As we can see, a stack of convolutional layers produces an equivariant mapping. In what follows, we will try to find out why convolutions have been chosen (by Y. LeCun, and others independently) to obtain this property.

Why do we use convolutions to get translation equivariance?

Let’s first imagine that we do not know what convolutions are. Our only objective is to add shift-equivariance to a neural network, since we just learned about this interesting property. We thus search for a neural network layer which is shift-equivariant.

Let’s also add the additional constraint of linearity. This is of course not a real requirement, since neural network layers can very well be non-linear, as long as they are differentiable (and even non-differentiable ones have traditionally been used, but this leads to approximations). However, the class of non-linear operators is extremely large, has many different functional forms, and it is difficult to find a class which can be easily parameterized such that it can represent universal functions (equivariant ones in our case). In classical MLPs, this problem is solved by restricting individual layers to be linear, followed by pointwise non-linearities (ReLU, tanh, sigmoid etc.). The universal approximation theorem shows that this class of functions can approximate any smooth function under mild assumptions.

The following derivation is taken from the book “Digital Image Processing”, by Bernd Jähne, Springer 1997. Yes, the book is quite old (but excellent), completely unrelated to neural works and machine learning … yet very relevant as we will see.

We thus suppose the existence of a new neural network layer, given as an operator ϕ on images / feature maps f, and which has the following properties:

• It is linear, i.e. given scalars α and β and images f and f’,

• It is shift-equivariant, i.e. given a shift operator m,nS, which shifts images f m pixels vertically and n pixels horizontally,

• It has a an impulse response h, which is not really a contraint. It just means that we know the result when the operator is applied to a Dirac impulse 0,0p centered at the origin (0,0):

We can first express the image/signal f as a linear combination of Dirac impulses p at different pixel locations. In the discrete case (which images are) this is always possible without losing any precision:

We use the linearity property of the operator to move it into the sums:

We can then replace the Dirac impulses at different positions with a single Dirac impuls at position (0,0), which is then for each pixel shifted by an amount (m,n) in y and x direction corresponding to the position of the pixel:

We can use the required shift-equivariance property of our operator ϕ to move it over the shift operator S:

Then, ϕ(0,0 p) is nothing else then the “impulse response”, so the output of our operator when applied to a Dirac at the origin, which we defined as being “h”. This is just a change of notation:

We can now express the result of the shift operator by integrating its effect into the position arguments of h:

Last, we can perform a change of variables, namely m’=x-m and n’=y-m, and we get

And this is …. a convolution operator which convolves the image f with a filter kernel h, which is equivalent to the impulse response of our operator.

This means, that the only linear and shift-equivariant operators are convolutions, and this is the reason we put convolutions into neural networks.

How about translation invariance?

While convolutions are translation equivariant and not invariant, an approximative translation invariance can be achieved in neural networks by combining convolutions with spatial pooling operators.

More information

If you are interested in more information, you can check out my lecture on convolutions, which is part of my lecture on Deep Learning and Differentiable Programming at INSA-Lyon, France.

Why is the result in the ICLR 2017 best paper “Understanding deep learning requires rethinking generalization” so interesting? (https://arxiv.org/abs/1611.03530)

Let’s think about model fitting and generalization using an example from Christopher Bishop’s 2006 book, namely fitting a polynomial to data sampled from a sine function (supposed to be unknown) :

We most frequently determine our model complexity (the “hypothesis class” / network architecture) using cross-validation on a hold-out set. Let’s suppose we have a hold-out set on the data above, not shown here. It would most likely lead to choosing a model / hypothesis class with hyper-parameter M=3, since it closely corresponds to the (unknown) function used to generate the (noisy) data :

We can expect this model to generalize reasonably well to a test set, since it already generalized well from the training to the hold-out set (ignoring here the fact that optimizing a model on a hold-out set can itself lead to overfitting).

Now, can we expect the same model / hypothesis class (polynomial with order M=3) to perform well on the same training data, where labels are replaced with random labels? In our case of fitting a polynomial to said data, this would involve pairs of (x,y), where the y-axis is random, e.g. this data :

The model will most probably not fit this semi-random data. The model M=3 leads to low overfitting on the original (not random) data since it is capable of expressing regularities in the data which arise from the marginal and the full data distribution, e.g. including labels. Replacing labels with random labels, removes regularities from the data. As a consequence, the model is not powerful enough to fit the data.

However, the authors in [https://arxiv.org/abs/1611.03530] showed that this is not the case for a large part of the deep models currently used in vision. Known models like VGG etc. can fit very large amounts of training data (like Imagenet) with random labels, although the same models, with the same capacity (the same number of parameters) have shown to generalize well from the training set to a hold-out set.

These results have been shown without regularization (no weight decay, not batch normalization, no dropout).