3 Methodology

Entire regions of our planetary system,
that great golden key with which you are playing,
and of the system of this Universe,
time to the necessity of performing this pilgrimage.
Would arrive at the correct solution,
face shews not the least wrinkle,
through his rash opinion of the improbability of performing,
faire ici le compte rendu technique de ma decouverte.

Acting upon this hint,
acted violently on my nervous system,
this was caused by intense heat acting on the organic matter of the earth.

The sum total of good playing,
and the Machine playing its large Wings,
that I would try it on myself acting forthwith on this decision.

This project combines research in science, art and the humanities—making it transdisciplinary.


Literature, Philosophy, Art


Cognitive Science, AI, DH


IR, NLP, Web Development

Traditional methodologies in these disciplines are very subject specific and a project combining elements of each field is left mixing and matching suitable methods from them all.

In this chapter I will outline the reasons why the existing intradisciplinary methodologies aren’t completely suitable for this project and then explain the choice of more transdisciplinary methods and how I combined them to suit my needs.

As mentioned in the Introduction the overall objectives of this project are to:

  1. Critically analyse and synthesise existing literature,

  2. develop pataphysical algorithms,

  3. design a system to demonstrate algorithms,

  4. develop a website as an artefact,

  5. define an evaluation and interpretation framework,

  6. analyse results.

Research methods that support these tasks are needed and I will address these four points again at the end of this chapter.

3.1 Intradisciplinary

Different disciplines prefer different research methodologies. Of the various disciplines that inform this research the specific subareas that are relevant are as follows.

3.1.1 Technology

Half of this project’s objectives are related to computer science therefore it is important to consider how research in this discipline is traditionally approached.

A framework for finding a suitable approach was suggested by Holz et al (2006). The following four steps form an iterative process. (1) “What do we want to achieve?” e.g. find out what is happening, develop something that works, evaluate an existing system/technology, compare existing systems, or change human behaviour. (2) “Where does the data come from?” e.g. how to collect? (read, observe, ask, measure, experiment, model) and where to collect? (field, laboratory, conceptual). (3) “What do we do with the data?”, e.g. identify themes/patterns/quotes, calculate numbers, identify trends, express via multimedia, create frameworks/taxonomies. (4) “Have we achieved our goal?” e.g. draw conclusions, evaluate results, or identify limitations.

Another option is to look at what computer science researchers have done historically. In a rather old but still insightful analysis of over 600 papers1 Ramesh et al (2004) have shown that—by far—the most common approach to research in computer science during this period was formulative with almost 79% use (as opposed to “descriptive” with 10% and “evaluative” with 11%). This was in particular in regards to “processes, methods and algorithms” which was used by just over 50% of researchers. Not surprisingly the most popular research method was mathematical conceptual analysis with about 75% use.

Jose Nelson Amaral (2006) classifies methodologies in computer science into five main categories as shown below.


Proof, verification, correctness


Testing, evaluation, question answering


Proof of concept, prototype, artefact


Understand and define processes


Abstraction, simulations

Here are this project’s answers to the four questions posed by Holz et al (2006).

What do we want to achieve?

- Understand human creativity and how this translates to machines.
- Understand the relationship of pataphysics and creativity.
- Understand how creativity is evaluated in humans and machines.
- Research suitable pataphysical concepts to be implemented as algorithms.
- Define algorithms formally.
- Implement prototype incorporating algorithms.
- Develop framework for interpreting and evaluating machine creativity.

Where does the data come from?

- Read pataphysical literature and research.
- Collate existing research on creativity and evaluation.
- Survey creative approaches to technology.
- Experiment with algorithms and implementation.

What do we do with the data?

- Iterate through developmental stages of algorithmic outputs.
- Create an artefact that represents the underlying philosophy and research.
- Create an evaluation framework based on theoretical research.

Have we achieved our goal?

- See conclusion chapter 14.

Referring back to the four objectives above, objective 1 is to create new creative search algorithms. This is not supposed to happen on a purely abstract basis but in a practical fashion (i.e. ‘experimental’), with a working implementation (i.e. ‘build’) as proof-of-concept (see objective 2). While the algorithms need to be defined in formal terms (i.e. ‘formal’), the goal here is not to create a theoretical proof of correctness (given the creative and rather subjective nature of the underlying philosophy this is virtually impossible) but a practical demonstration of the creative processes behind. Overall this would suggest an experimental approach with prototyping of an artefact. Objective is to come up with a suitable definition of creativity (i.e. ‘process’). This should be informed by existing research. Again, we are not interested in formulating this in mathematical terms and proofs but rather a more esoteric and systemic view. Because the definition needs to apply to humans and machines it needs to be precise enough. Objective is then to create an overall theoretical framework (i.e. ‘model’) for the evaluation of creativity in humans and machines.

By now we have managed to cover every one of the major methodologies mentioned by Amaral et al. (2006) but we are still lacking ways to address the subjective and creative nature of the project. Furthermore, the philosophical and artistic inspirations that inform the development of the artefact don’t get enough of a voice in these methods. In computer science, implementations are generally seen as a proof of concepts or prototypes—when really they should be seen as artefacts in the sense of artistic pieces of work. So, to really appreciate the scope of this practical element of this project we need to consider research in the arts and humanities too.

3.1.2 Arts and Humanities

A hallmark of humanistic study is that research is approached differently than in the natural and social sciences, where data and hard evidence are required to draw conclusions. Because the human experience cannot be adequately captured by facts and figures alone, humanities research employs methods that are historical, interpretive and analytical in nature.

(Standford n.d.)

Malins and Gray suggest the following ideas for arts-based researchers searching for the right methodology (1995).

  • Consider a range of research strategies (from all disciplines).

  • ‘Tailor’ the research to the nature of project and the researcher’s expertise.

  • Carry out the research from an informed perspective, as ‘participant observer’.

  • Continually define and refine the research question, allowing methodologies to emerge.

  • Acknowledge accessibility, discipline, rigour, transparency, and transferability.

  • Be aware of the critical context of practice and research and raise the level of critical debate.

  • Consider interdisciplinary / multidisciplinary approaches to research.

They further elaborate on the key characteristics of arts methodologies as follows (Gray and Malins 2004).

  • Experiencing/exploring, gathering, documenting information and generating data/evidence.

  • Reflecting on and evaluating information, selecting the most relevant information.

  • Analysing, interpreting and making sense of information.

  • Synthesizing and communicating research findings, planning new research.

(Gray and Malins 2004)

They further specify a whole set of individual methods used for the approaches above.

  • observation and related notation/use of symbols

  • visualization

  • drawing (in all forms)

  • diagrams

  • concept mapping, mind mapping

  • brainstorming/lateral thinking

  • sketchbook/notebook

  • photography, video, audio

  • 3D models/maquettes

  • experimentation with materials and processes

  • modelling/simulations

  • multimedia/hypermedia applications

  • digital databases, visual and textual glossaries and archives

  • reflection-in-action/‘stream of consciousness’/personal narrative

  • visual diary/reflective journal/research diary

  • collaboration/participation/feedback, for example workshops

  • use of metaphor and analogy

  • organizational and analytical matrices

  • decision-making flow charts

  • story boards, visual narratives

  • curation

  • critical writing, publications

  • exposition and peer feedback/review

(Gray and Malins 2004)

The discpiline of Digital Humanities (DH) (see chapter 5.3.4) seems like a logical choice to look for suitable methodologies. It is characterised by “collaboration, transdisciplinarity and an engagement with computing” (Burdick et al. 2012) but it should not simply be reduced to “doing the humanities digitally” (2012). Transliteracy, an understanding of several kinds of tools and media, is an important aspect in this (Thomas et al. 2007). DH can be broken down into the following set of methodologies.


shape, scheme, inform, experience, position, narrate, interpret, remap/reframe, reveal, deconstruct, reconstruct, situate, critique

Curation, analysis, editing, modelling

digitise, classify, describe, metadata, organise, navigate

Computation, processing

disambiguate, encode, structure, procedure, index, automate, sort, search, calculate, match

Networks, infrastructure

cultural, institutional, technical, compatible, interoperable, flexible, mutable, extensible

Versioning, prototyping, failures

iterate, experiment, take-risks, redefine, beta-test

Some of the emerging research methods Burdick et al. have identified are listed below (2012) (The full list can be found in appendix A.3).

  • structured mark-up

  • natural language processing

  • mutability

  • digital cultural record

  • algorithmic analysis

  • distant/close, macro/micro, surface/depth

  • parametrics

  • cultural mash-ups

  • algorithm design

  • data visualization

  • modelling knowledge

  • ambient data

  • collaborative authorship

  • interdisciplinary teams

  • use as performance

  • narrative structures

  • code as text

  • software in a cultural context

  • repurposable content and remix culture

  • participatory web

  • read/write/rewrite

  • meta-medium

  • polymorphous browsing

Several of the methodologies listed by Gray and Malins (2004) seem to apply to the research presented in this thesis. Exploring, evaluating, analysing, interpreting, synthesising and disseminating research all are part of it. However, looking at the specific methods they collated, the difference becomes clearer as only the following 7 appear relevant (visualization, experimentation with processes, multimedia/hypermedia applications, use of metaphor and analogy, organizational and analytical matrices, curation, and critical writing, publications).

The DH methodologies seem more useful. In terms of design, pata.physics.wtf positions itself in context and the evaluation framework interprets and critiques AMC. Before that I curate the two corpora, digitise them and organise them. Computing comes in at verious stages, to (dis)ambiguate (i.e. pataphysicalise), encode, index, search and match data. The infrastructure is cultural, technical and extensible, relying on the WWW for several spects. Versioning, prototyping and failures all come in during the iterative development process, which involves a lot of experimentation and refactoring. Furthermore, the research methods Burdick et al (2012) list match this project much better (although of course the list above was already only a selection that was deemed relevant; the original list was much larger. See appendix A.3).

3.2 Transdisciplinary

Nicolescu distinguished between 3 different kinds of research “without stable boundaries between the disciplines.”2 (2010).


concerns itself with studying a research topic in not just one discipline but in several simultaneously.


concerns the transfer of methods from one discipline to another.


concerns that which is at once between the disciplines, across the different disciplines, and beyond all disciplines.

The standard epistemological view of science and art is that they are objective and subjective, respectively. So, what does that mean for research conducted between, across and beyond science and art, i.e. research that is transdisciplinary?

Nicolescu criticised the view that science must be objective. He even claimed that any non-scientific knowledge is “cast into the inferno of subjectivity, tolerated at most as a meaningless embellishment or rejected with contempt as a fantasy, an illusion, a regression, or a product of the imagination” (2010). Objectivity, he said, becomes the “supreme criterion of Truth”3

The death of the Subject is the price we pay for objective knowledge.

(Nicolescu 2010)

He went on to quote Werner Heisenberg on the concepts of objective and subjective reality: “we would make a very crude simplification if we want to divide the world in[to] one objective reality and one subjective reality. Many rigidities of the philosophy of the last centuries are born by this black and white view of the world” (Heisenberg, cited in Nicolescu 2010).

The too strong insistence on the difference between scientific knowledge and artistic knowledge comes from the wrong idea that concepts describe perfectly the ‘real things’. […] All true philosophy is situated on the threshold between science and poetry.4

(Heisenberg, cited in Nicolescu 2010)

In transdisciplinarity traditional disciplinary boundaries have no meaning.

Nicolescu's transdisciplinarity
Figure 3.1 - Nicolescu's transdisciplinarity

Working across disciplines requires a new unique methodology. Nicolescu proposed a methodology of transdisciplinarity as a non-hierarchical ternary partition of ‘Subject, Object and Hidden Third’ (as shown in figure [fig:trans]) rather than the traditional binary partition of ‘Subject versus Object’ (2010).

The old principle “unity in diversity and diversity from unity” is embodied in transdisciplinarity.’

(Nicolescu 2010)

‘unite and conquer’ ‘divide and conquer’

(Yang 2013)

Hugill and Yang agree that existing research methodologies are unsuitable for transdisciplinary subjects such as Creative Computing (CC). The following is an example of a possible CC research methodology they propose as a starting point (Hugill and Yang 2013):

  1. Review literature across disciplines.

  2. Identify key creative activities.

  3. Analyse the processes of creation.

  4. Propose approaches to support these activities and processes.

  5. Design and implement software following this approach.

  6. Experiment with the resulting system and propose framework.

They go on to propose four standards for CC (Hugill and Yang 2013) namely, (1) resist standardisation, (2) perpetual novelty, (3) continuous user interaction and (4) combinational, exploratory and or transformational.

A different model was suggested by Edmonds and Candy in their Trajectory Model of Practice and Research (TMPR), a framework to “influence practice, inform theory and, in particular, shape evaluation” (2010). Figure 3.2 shows the TMPR which allows for different trajectories between practice, theory and evaluation. Table 3.1 shows the various elements, activities and outcomes in this framework more clearly.

Edmonds and Candy's trajectory model (TMPR)
Figure 3.2 - Edmonds and Candy's trajectory model (TMPR)
Elements Activities Outcomes
Practice create, exhibit, reflect Works: consisting of physical artefacts, musical compositions, software systems, installations, exhibitions, collaborations
Theory read, think, write, develop Frameworks: comprising questions, criteria, issues
Evaluation observe, record, analyse, reflect Results: findings leading to new/modified Works and Frameworks
Table 3.1 Elements, activities and outcomes of each trajectory in the TMPR

This project positions itself “at once between the disciplines, across the different disciplines, and beyond all disciplines”—making it transdisciplinary. The abolishment of disciplinary boundaries suits the unique context of this research. Pataphysics specifically is highly subjective. Searle highlighted that ontologically subjective topics (such as creativity) can be studied in epistemically objective ways (2015), which, as doctoral research, this project attempts to do.

The Hugill and Yang methodology seems general enough to fit the needs of this project, with all 6 points covered in the various chapters of this thesis.

  1. Review literature across disciplines (chapters 2, 4, 5, 6, and 7).

  2. Define creativity in humans and machines (chapters 456 and 7).

  3. Analyse the relation between the disciplines above (chapter 8).

  4. Propose algorithms to support creativity in machines (chapter 10).

  5. Design and implement software following this approach (chapter 10).

  6. Experiment with the resulting system and propose interpretation/evaluation framework (chapters 12, 13, and 9).

Figure 3.3 shows how the TMPR could be applied to this project.

3.3 Patadisciplinary

So, to summarise, this project draws from several different disciplines as mentioned at the beginning of this chapter: pataphysics—literture, philosophy, art, creativity—cognitive science, AI, DH, and technology—IR, NLP, web development.


Transdisciplinary, subjective


Creative computing, exploratory, experimental


Artefact, literature synthesis, algorithm design, theoretical framework, critical reflection and analysis, rapid incremental prototyping

The general workflow of this project was as follows: (1) critically analyse and synthesise existing literature, (2) develop pataphysical algorithms, (3) design a system to demonstrate algorithms, (4) develop a website as an artefact, (5) define an evaluation and interpretation framework, and (6) analyse results.

This project's trajectory model
Figure 3.3 - This project's trajectory model

As figure 3.3 shows, the practive trajectory of this research is based on the practical development of a website to contain the exploratory search tool and implementation of the theoreical algorithms. The theory trajectory is about defining those algorithms formally in historic and topical context based on a critical survey of related literature. This also includes the development of a theoretical framework for the evaluation and interpretation of creative artefacts. The Evaluation trajectory then is all about the results. That includes an analysis of the work completed. The arrows in the figure indicate how these different trajectories influence each other.


Amaral, Jose Nelson, Michael Buro, Renee Elio, Jim Hoover, Ioanis Nikolaidis, Mohammad Salavatipour, Lorna Stewart, and Ken Wong. 2006. “About Computing Science Research Methodology.” University of Alberta. link.

Burdick, Anne, Johanna Drucker, Peter Lunefeld, Todd Presner, and Jeffrey Schnapp. 2012. Digital Humanities. Cambridge, MA, USA: MIT Press.

Edmonds, Ernest, and Linda Candy. 2010. “Relating Theory, Practice and Evaluation in Practitioner Research.” Leonardo 43 (5): 470–76.

Gray, Carole, and Julian Malins. 2004. Visualizing research: a guide to the research process in art and design. New York, USA: Routledge.

Holz, Hilary J, Anne Applin, Donald Joyce, Helen Purchase, Catherine Reed, and Bruria Haberman. 2006. “Research Methods in Computing: What are they, and how should we teach them ?” Innovation and Technology in Computer Science Education, 96–114.

Hugill, Andrew, and Hongji Yang. 2013. “The creative turn: new challenges for computing.” International Journal of Creative Computing 1 (1): 4–19.

Malins, Julian, and Carole Gray. 1995. “Appropriate research methodologies for artists, designers and craftspersons: research as a learning process.” In “Making It”: UK Crafts Council Conference.

Nicolescu, Basarab. 2010. “Methodology of Transdisciplinarity - Levels of Reality, Logic of the Included.” Transcdisciplinary Journal of Engineering and Science 1 (1): 19–38.

Ramesh, V., Robert L. Glass, and Iris Vessey. 2004. “Research in computer science: an empirical study.” Systems and Software 70 (1-2): 165–76.

Searle, John. 2015. “Consciousness in Artificial Intelligence.” Talks at Google. link.

Thomas, Sue, Chris Joseph, Jess Laccetti, Bruce Mason, Simon Mills, Simon Perril, and Kate Pullinger. 2007. “Transliteracy: Crossing divides.” First Monday 12 (12).

  1. While the paper itself was published in 2004, the body of work was based on publications from between 1995 and 1999—this suggests that a lot of the more “recent” research around web technologies is not included in this study.

  2. Nicolescu cites Jean Piaget here, who first coined the term ‘transdisciplinarity’ in 1972.

  3. As we shall see later, pataphysics does the opposite: it reveres the Subject.

  4. The full paragraph is worth quoting—see appendix A.2.