Let $K$ be a field and $GL_n(K)$ the group of invertible n by n matrices over $K$. About a century ago, I. Schur described the irreducible polynomial representations of $GL_n(K)$ for the field $K$ of complex numbers. The same problem over a field $K$ of prime characteristic is still widely open. In this context, G. James proved in 1981 a so-called column removal formula. It relates polynomial representations in different homogeneous degrees. Combinatorially, it is based on removing the first column of an associated Young diagram. We generalize this operation of column removal to considering ``complements" of Young diagrams. Algebraically this establishes equivalences between categories of polynomial representations in different degrees. As by-product it establishes and explains certain repeating patterns in decomposition numbers of general linear and symmetric groups. This work is related to work by Beilinson-Lusztig and MacPherson. It is joint work with Fang and Koenig.