Skip to Content

Department of Cognitive Science

Measuring representation

Lars Marstaller (
Macquarie Centre for Cognitive Science, Macquarie University, Sydney
Arend Hintze (
Keck Graduate Institute for Applied Life Sciences, Claremont, CA
Christoph Adami (
Keck Graduate Institute for Applied Life Sciences, Claremont, CA


We present a measure of representation in neural networks that we call ‘R’, which is based on information theory. We show how R relates to an analysis of distributed representation, viz. a principal components analysis of activation space. Finally, we argue that R is well suited to measure representation in neural networks.

Citation details for this article:

Marstaller, L., Hintze, A., Adami, C. (2010). Measuring representation. In W. Christensen, E. Schier, and J. Sutton (Eds.), ASCS09: Proceedings of the 9th Conference of the Australasian Society for Cognitive Science (pp. 232-237). Sydney: Macquarie Centre for Cognitive Science.

DOI: 10.5096/ASCS200935
Download the PDF here


  1. Adami, C. & Cerf, N. J. (2000) Physical complexity of symbolic sequences. Physica D, 120.
  2. Brooks, R. A. (1991). Intelligence without representation. Artificial Intelligence, 47, 139–159.
  3. Churchland, P. & Sejnowski, T. J. (1992). The computational brain. Cambridge, MA: MIT Press.
  4. Clark, A. (1993) Associative engines. Connectionism, concepts, and representational change. Cambridge, MA: MIT Press.
  5. Clark, A. (1997). Being there: putting brain, body, and world together again. Cambridge, MA: MIT Press.
  6. Elman, J. L. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195-225.
  7. O’Brien, G. & Opie, J. (2009). The role of representation in computation. Cognitive Processing, 10, 53-62.
  8. Pylyshyn, Z. (1984). Computation and cognition: toward a foundation for cognitive science. Cambridge, MA: MIT Press.
  9. Shannon, C. E. (1948) A mathematical theory of communication. Bell System Technical Journal, 27, 379–423 & 623–656.
  10. Smolensky, P. (1987). Connectionist AI, symbolic AI, and the brain. Artificial Intelligence Review, 1, 95-109.
  11. Thornton, C. (2009) Representation recovers information. Cognitive Science 33(8), 1-30.

Further Information

Who is Visiting

Contact Details

Telephone: (02) 9850 9599
Fax : (02) 9850 6059
Email :
Web :