publications

2023

1. Proceedings

   On the Entropy of Rectifiable and Stratified Measures

   Juan Pablo Vigneaux

   In Nielsen, Frank and Barbaresco, Frédéric, Geometric Science of Information: 6th International Conference, GSI 2023, Lecture Notes in Computer Science, 14071, 338–346, 2023

   Abs arXiv Website

   We summarize some results of geometric measure theory concerning rectifiable sets and measures. Combined with the entropic chain rule for disintegrations (Vigneaux, 2021), they account for some properties of the entropy of rectifiable measures with respect to the Hausdorff measure first studied by (Koliander et al., 2016). Then we present some recent work on stratified measures, which are convex combinations of rectifiable measures. These generalize discrete-continuous mixtures and may have a singular continuous part. Their entropy obeys a chain rule, whose “conditional term” is an average of the entropies of the rectifiable measures involved. We state an asymptotic equipartition property (AEP) for stratified measures that shows concentration on strata of a few “typical dimensions” and that links the conditional term of the chain rule to the volume growth of typical sequences in each stratum.

2. Proceedings

   Categorical magnitude and entropy

   Stephanie Chen, and Juan Pablo Vigneaux

   In Nielsen, Frank and Barbaresco, Frédéric, Geometric Science of Information: 6th International Conference, GSI 2023, Lecture Notes in Computer Science, 14071, 278-287, 2023

   Abs arXiv Website

   Given any finite set equipped with a probability measure, one may compute its Shannon entropy or information content. The entropy becomes the logarithm of the cardinality of the set when the uniform probability is used. Leinster introduced a notion of Euler characteristic for certain finite categories, also known as magnitude, that can be seen as a categorical generalization of cardinality. This paper aims to connect the two ideas by considering the extension of Shannon entropy to finite categories endowed with probability, in such a way that the magnitude is recovered when a certain choice of "uniform" probability is made.

3. Journal

   Typicality for stratified measures

   Juan Pablo Vigneaux

   IEEE Transactions on Information Theory, 2023

   Abs arXiv Website

   We define \emphstratified measures on Euclidean space as convex combinations of rectifiable measures. They are possibly singular with respect to the Lebesgue measure and generalize discrete-continuous mixtures. A stratified measure ρcan thus be represented as \sum_i=1^k q_i \rho_i, where (q_1,..,q_k) is a probability vector and each \rho_i is absolutely continuous with respect to the m_i-Hausdorff measure \mu_i on a m_i-rectifiable set E_i (e.g. a smooth m_i-manifold if m_i>0 or a countable set if m_i=0). We introduce a set of strongly typical realizations of ρ^⊗n that occur with high probability; they are supported on a finite union of strata E_i_1\times ⋯\times E_i_n whose dimensions “per factor” concentrate around the \emphmean dimension \sum_i=1^k q_i m_i. For each n, an appropriate sum of Hausdorff measures on the different strata gives a notion of reference “volume”; the exponential growth rate of the typical set’s volume is quantified by Csiszar’s \emphgeneralized entropy of ρwith respect to μ=\sum_i=1^k \mu_i. This entropy satisfies a chain rule; the conditional term is related to the volume growth of the typical realizations in each stratum. Under suitable hypotheses, our notion of mean dimension coincides with Rényi’s information dimension when applied to stratified measures, but the generalized entropy used here differs from Rényi’s dimensional entropy.

4. Journal

   A characterization of generalized multinomial coefficients related to the entropic chain rule

   Juan Pablo Vigneaux

   Aequationes mathematicae, 2023

   Abs arXiv Website

   There is an asymptotic correspondence between the multiplicative relations among multinomial coefficients and the (additive) recursive property of Shannon entropy known as the chain rule. We show that both types of identities are manifestations of a unique algebraic construction: a 1-cocycle condition in *information cohomology*, an algebraic invariant of presheaves of modules on certain categories of observables. Depending on the coefficients, the 1-cocycles can be information measures (Shannon entropy, Tsallis α-entropy) or generalized (Fontené-Ward) multinomial coefficients. In each case the 1-cocycle condition encodes a system of functional equations. We obtain in particular a combinatorial analogue of the “fundamental equation of information theory”: a simple functional equation that uniquely characterizes the generalized binomial coefficients. The asymptotic correspondence mentioned above extends to any α-entropy and certain multinomial coefficients with compatible asymptotic behavior, shedding new light on the meaning of the chain rule and its deformations.

5. Journal

   A formula for the categorical magnitude in terms of the Moore-Penrose pseudoinverse

   Stephanie Chen, and Juan Pablo Vigneaux

   Bulletin of the Belgian Mathematical Society - Simon Stevin, 2023

   To appear in Vol. 30, Issue 3

   Abs arXiv

   The magnitude of finite categories is a generalization of the Euler characteristic. It is defined using the coarse incidence algebra of rational-valued functions on the given finite category, and a distinguished element in this algebra: the Dirichlet zeta function. The incidence algebra may be identified with the algebra of n\times n matrices over the rational numbers, where n is the cardinality of the underlying object set. The Moore-Penrose pseudoinverse of a matrix is a generalization of the inverse; it exists and is unique for any given matrix over the complex numbers. In this article, we derive a new method for calculating the magnitude of a finite category, using the pseudoinverse of the matrix that corresponds to the zeta function. The magnitude equals the sum of the entries of this pseudoinverse.

2021

1. Proceedings

   Entropy under disintegrations

   Juan Pablo Vigneaux

   In Nielsen, Frank and Barbaresco, Frédéric, Geometric Science of Information: 5th International Conference, GSI 2021, Lecture Notes in Computer Science, 12829, 340–349, 2021

   Abs arXiv Slides Website

   We consider the differential entropy of probability measures absolutely continuous with respect to a given σ-finite reference measure on an arbitrary measurable space. We state the asymptotic equipartition property in this general case; the result is part of the folklore but our presentation is to some extent novel. Then we study a general framework under which such entropies satisfy a chain rule: disintegrations of measures. We give an asymptotic interpretation for conditional entropies in this case. Finally, we apply our result to Haar measures in canonical relation.

2. Proceedings

   Information cohomology of classical vector-valued observables

   Juan Pablo Vigneaux

   In Nielsen, Frank and Barbaresco, Frédéric, Geometric Science of Information: 5th International Conference, GSI 2021, Lecture Notes in Computer Science, 12829, 537–546, 2021

   Abs arXiv Slides Website

   We provide here a novel algebraic characterization of two information measures associated with a vector-valued random variable, its differential entropy and the dimension of the underlying space, purely based on their recursive properties (the chain rule and the nullity-rank theorem, respectively). More precisely, we compute the information cohomology of Baudot and Bennequin with coefficients in a module of continuous probabilistic functionals over a category that mixes discrete observables and continuous vector-valued observables, characterizing completely the 1-cocycles; evaluated on continuous laws, these cocycles are linear combinations of the differential entropy and the dimension.

2020

1. Preprint

   Extra-fine sheaves and interaction decompositions

   Daniel Bennequin, Olivier Peltre, Grégoire Sergeant-Perthuis, and Juan Pablo Vigneaux

   arXiv preprint 2009.12646, 2020

   Abs arXiv

   We introduce an original notion of extra-fine sheaf on a topological space, and a variant (hyper-extra-fine) for which Čech cohomology in strictly positive degree vanishes. We provide a characterization of such sheaves when the topological space is a partially ordered set (poset) equipped with the Alexandrov topology. Then we further specialize our results to some sheaves of vector spaces and injective maps, where extra-fineness is (essentially) equivalent to the decomposition of the sheaf into a direct sum of subfunctors, known as interaction decomposition, and can be expressed by a sum-intersection condition. We use these results to compute the dimension of the space of global sections when the presheaves are freely generated over a functor of sets, generalizing classical counting formulae for the number of solutions of the linearized marginal problem (Kellerer and Matúš). We finish with a comparison theorem between the Čech cohomology associated to a covering and the topos cohomology of the poset with coefficients in the presheaf, which is also the cohomology of a cosimplicial local system over the nerve of the poset. For that, we give a detailed treatment of cosimplicial local systems on simplicial sets. The appendixes present presheaves, sheaves and Čech cohomology, and their application to the marginal problem.

2. Journal

   A functional equation related to generalized entropies and the modular group

   Daniel Bennequin, and Juan Pablo Vigneaux

   Aequationes mathematicae, 2020

   Abs arXiv Website

   We solve a functional equation connected to the algebraic characterization of generalized information functions. To prove the symmetry of the solution, we study a related system of functional equations, which involves two homographies. These transformations generate the modular group, and this fact plays a crucial role in solving the system. The method suggests a more general relation between conditional probabilities and arithmetic.

3. Journal

   Information structures and their cohomology

   Juan Pablo Vigneaux

   Theory and Applications of Categories, 2020

   Abs arXiv Website

   We introduce the category of information structures, whose objects are suitable diagrams of measurable sets that encode the possible outputs of a given family of observables and their mutual relationships of refinement; they serve as mathematical models of contextuality in classical and quantum settings. Each information structure can be regarded as a ringed site with trivial topology; the structure ring is generated by the observables themselves and its multiplication corresponds to joint measurement. We extend Baudot and Bennequin’s definition of information cohomology to this setting, as a derived functor in the category of modules over the structure ring, and show explicitly that the bar construction gives a projective resolution in that category, recovering in this way the cochain complexes previously considered in the literature. Finally, we study the particular case of a one-parameter family of coefficients made of functions of probability distributions. The only 1-cocycles are Shannon entropy or Tsallis α-entropy, depending on the value of the parameter.

2019

1. Thesis

   Topology of statistical systems: a cohomological approach to information theory

   Juan Pablo Vigneaux

   Université Sorbonne Paris Cité, 2019

   Abs PDF Slides Website

   This thesis extends in several directions the cohomological study of information theory pioneered by Baudot and Bennequin. We introduce a topos-theoretical notion of statistical space and then study several cohomological invariants. Information functions and related objects appear as distinguished cohomology classes; the corresponding cocycle equations encode recursive properties of these functions. Information has thus topological meaning and topology serves as a unifying framework. Part I discusses the geometrical foundations of the theory. Information structures are introduced as categories that encode the relations of refinement between different statistical observables. We study products and coproducts of information structures, as well as their representation by measurable functions or hermitian operators. Every information structure gives rise to a ringed site; we discuss in detail the definition of \emphinformation cohomology using the homological tools developed by Artin, Grothendieck, Verdier and their collaborators. Part II studies the cohomology of discrete random variables. Information functions—Shannon entropy, Tsallis α-entropy, Kullback-Leibler divergence—appear as 1-cocycles for appropriate modules of probabilistic coefficients (functions of probability laws). In the combinatorial case (functions of histograms), the only 0-cocycle is the exponential function, and the 1-cocycles are generalized multinomial coefficients (Fontené-Ward). There is an asymptotic relation between the combinatorial and probabilistic cocycles. Part III studies in detail the q-multinomial coefficients, showing that their growth rate is connected to Tsallis 2-entropy (quadratic entropy). When q is a prime power, these q-multinomial coefficients count flags of finite vector spaces with prescribed length and dimensions. We obtain a combinatorial explanation for the nonadditivity of the quadratic entropy and a frequentist justification for the maximum entropy principle with Tsallis statistics. We introduce a discrete-time stochastic process associated to the q-binomial probability distribution that generates finite vector spaces (flags of length 2). The concentration of measure on certain \emphtypical subspaces allows us to extend Shannon’s theory to this setting. Part IV discusses the generalization of information cohomology to continuous random variables. We study the functoriality properties of conditioning (seen as disintegration) and its compatibility with marginalization. The cohomological computations are restricted to the real valued, gaussian case. When coordinates are fixed, the 1-cocycles are the differential entropy as well as generalized moments. When computations are done in a coordinate-free manner, with the so-called \emphgrassmannian categories, we recover as the only degree-one cohomology classes the entropy and the dimension. This constitutes a novel algebraic characterization of differential entropy.

2. Journal

   Information theory with finite vector spaces

   Juan Pablo Vigneaux

   IEEE Transactions on Information Theory, 2019

   Abs arXiv Slides Website

   Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their q-deformation; when q is a prime power, these q-multinomial coefficients count flags of finite vector spaces with prescribed length and dimensions. In particular, the q-binomial coefficients count vector subspaces of given dimension. We obtain in this way a combinatorial explanation for the nonadditivity of the quadratic entropy, which arises from a recursive counting of flags. We show that statistical systems whose configurations are described by flags provide a frequentist justification for the maximum entropy principle with Tsallis statistics. We introduce then a discrete-time stochastic process associated to the q-binomial probability distribution, that generates at time n a vector subspace of \mathbb F_q^n (here \mathbb F_q is the finite field of order q). The concentration of measure on certain "typical subspaces" allows us to extend the asymptotic equipartition property to this setting. The size of the typical set is quantified by the quadratic entropy. We discuss the applications to Shannon theory, particularly to source coding, when messages correspond to vector spaces.