3 Methodology
Entire regions of our planetary system,that great golden key with which you are playing,and of the system of this Universe,time to the necessity of performing this pilgrimage. Would arrive at the correct solution,face shews not the least wrinkle,through his rash opinion of the improbability of performing,faire ici le compte rendu technique de ma decouverte.Acting upon this hint,acted violently on my nervous system,this was caused by intense heat acting on the organic matter of the earth.The sum total of good playing,and the Machine playing its large Wings,that I would try it on myself acting forthwith on this decision.
This project combines research in science, art and the humanities—making it transdisciplinary.
- Pataphysics
Literature, Philosophy, Art
- Creativity
Cognitive Science, AI, DH
- Technology
IR, NLP, Web Development
Traditional methodologies in these disciplines are very subject specific and a project combining elements of each field is left mixing and matching suitable methods from them all.
In this chapter I will outline the reasons why the existing intradisciplinary methodologies aren’t completely suitable for this project and then explain the choice of more transdisciplinary methods and how I combined them to suit my needs.
As mentioned in the Introduction the overall objectives of this project are to:
Critically analyse and synthesise existing literature,
develop pataphysical algorithms,
design a system to demonstrate algorithms,
develop a website as an artefact,
define an evaluation and interpretation framework,
analyse results.
Research methods that support these tasks are needed and I will address these four points again at the end of this chapter.
3.1 Intradisciplinary
Different disciplines prefer different research methodologies. Of the various disciplines that inform this research the specific subareas that are relevant are as follows.
Information Retrieval
Interface Design
Web Development
Poetry, Literature, and Art
Philosophy
Human and Machine Creativity
Creative Computing
Computational Creativity
3.1.1 Technology
Half of this project’s objectives are related to computer science therefore it is important to consider how research in this discipline is traditionally approached.
A framework for finding a suitable approach was suggested by Holz et al (2006). The following four steps form an iterative process. (1) “What do we want to achieve?” e.g. find out what is happening, develop something that works, evaluate an existing system/technology, compare existing systems, or change human behaviour. (2) “Where does the data come from?” e.g. how to collect? (read, observe, ask, measure, experiment, model) and where to collect? (field, laboratory, conceptual). (3) “What do we do with the data?”, e.g. identify themes/patterns/quotes, calculate numbers, identify trends, express via multimedia, create frameworks/taxonomies. (4) “Have we achieved our goal?” e.g. draw conclusions, evaluate results, or identify limitations.
Another option is to look at what computer science researchers have done historically. In a rather old but still insightful analysis of over 600 papers1 Ramesh et al (2004) have shown that—by far—the most common approach to research in computer science during this period was formulative with almost 79% use (as opposed to “descriptive” with 10% and “evaluative” with 11%). This was in particular in regards to “processes, methods and algorithms” which was used by just over 50% of researchers. Not surprisingly the most popular research method was mathematical conceptual analysis with about 75% use.
Jose Nelson Amaral (2006) classifies methodologies in computer science into five main categories as shown below.
- Formal
Proof, verification, correctness
- Experimental
Testing, evaluation, question answering
- Build
Proof of concept, prototype, artefact
- Process
Understand and define processes
- Model
Abstraction, simulations
Here are this project’s answers to the four questions posed by Holz et al (2006).
- What do we want to achieve?
- Understand human creativity and how this translates to machines.
- Understand the relationship of pataphysics and creativity.
- Understand how creativity is evaluated in humans and machines.
- Research suitable pataphysical concepts to be implemented as algorithms.
- Define algorithms formally.
- Implement prototype incorporating algorithms.
- Develop framework for interpreting and evaluating machine creativity.- Where does the data come from?
- Read pataphysical literature and research.
- Collate existing research on creativity and evaluation.
- Survey creative approaches to technology.
- Experiment with algorithms and implementation.- What do we do with the data?
- Iterate through developmental stages of algorithmic outputs.
- Create an artefact that represents the underlying philosophy and research.
- Create an evaluation framework based on theoretical research.- Have we achieved our goal?
- See conclusion chapter 14.
Referring back to the four objectives above, objective 1 is to create new creative search algorithms. This is not supposed to happen on a purely abstract basis but in a practical fashion (i.e. ‘experimental’), with a working implementation (i.e. ‘build’) as proof-of-concept (see objective 2). While the algorithms need to be defined in formal terms (i.e. ‘formal’), the goal here is not to create a theoretical proof of correctness (given the creative and rather subjective nature of the underlying philosophy this is virtually impossible) but a practical demonstration of the creative processes behind. Overall this would suggest an experimental approach with prototyping of an artefact. Objective is to come up with a suitable definition of creativity (i.e. ‘process’). This should be informed by existing research. Again, we are not interested in formulating this in mathematical terms and proofs but rather a more esoteric and systemic view. Because the definition needs to apply to humans and machines it needs to be precise enough. Objective is then to create an overall theoretical framework (i.e. ‘model’) for the evaluation of creativity in humans and machines.
By now we have managed to cover every one of the major methodologies mentioned by Amaral et al. (2006) but we are still lacking ways to address the subjective and creative nature of the project. Furthermore, the philosophical and artistic inspirations that inform the development of the artefact don’t get enough of a voice in these methods. In computer science, implementations are generally seen as a proof of concepts or prototypes—when really they should be seen as artefacts in the sense of artistic pieces of work. So, to really appreciate the scope of this practical element of this project we need to consider research in the arts and humanities too.
3.1.2 Arts and Humanities
A hallmark of humanistic study is that research is approached differently than in the natural and social sciences, where data and hard evidence are required to draw conclusions. Because the human experience cannot be adequately captured by facts and figures alone, humanities research employs methods that are historical, interpretive and analytical in nature.
(Standford n.d.)
Malins and Gray suggest the following ideas for arts-based researchers searching for the right methodology (1995).
Consider a range of research strategies (from all disciplines).
‘Tailor’ the research to the nature of project and the researcher’s expertise.
Carry out the research from an informed perspective, as ‘participant observer’.
Continually define and refine the research question, allowing methodologies to emerge.
Acknowledge accessibility, discipline, rigour, transparency, and transferability.
Be aware of the critical context of practice and research and raise the level of critical debate.
Consider interdisciplinary / multidisciplinary approaches to research.
They further elaborate on the key characteristics of arts methodologies as follows (Gray and Malins 2004).
(Gray and Malins 2004)
Experiencing/exploring, gathering, documenting information and generating data/evidence.
Reflecting on and evaluating information, selecting the most relevant information.
Analysing, interpreting and making sense of information.
Synthesizing and communicating research findings, planning new research.
They further specify a whole set of individual methods used for the approaches above.
(Gray and Malins 2004)
observation and related notation/use of symbols
visualization
drawing (in all forms)
diagrams
concept mapping, mind mapping
brainstorming/lateral thinking
sketchbook/notebook
photography, video, audio
3D models/maquettes
experimentation with materials and processes
modelling/simulations
multimedia/hypermedia applications
digital databases, visual and textual glossaries and archives
reflection-in-action/‘stream of consciousness’/personal narrative
visual diary/reflective journal/research diary
collaboration/participation/feedback, for example workshops
use of metaphor and analogy
organizational and analytical matrices
decision-making flow charts
story boards, visual narratives
curation
critical writing, publications
exposition and peer feedback/review
The discpiline of Digital Humanities (DH) (see chapter 5.3.4) seems like a logical choice to look for suitable methodologies. It is characterised by “collaboration, transdisciplinarity and an engagement with computing” (Burdick et al. 2012) but it should not simply be reduced to “doing the humanities digitally” (2012). Transliteracy, an understanding of several kinds of tools and media, is an important aspect in this (Thomas et al. 2007). DH can be broken down into the following set of methodologies.
- Design
shape, scheme, inform, experience, position, narrate, interpret, remap/reframe, reveal, deconstruct, reconstruct, situate, critique
- Curation, analysis, editing, modelling
digitise, classify, describe, metadata, organise, navigate
- Computation, processing
disambiguate, encode, structure, procedure, index, automate, sort, search, calculate, match
- Networks, infrastructure
cultural, institutional, technical, compatible, interoperable, flexible, mutable, extensible
- Versioning, prototyping, failures
iterate, experiment, take-risks, redefine, beta-test
Some of the emerging research methods Burdick et al. have identified are listed below (2012) (The full list can be found in appendix A.3).
structured mark-up
natural language processing
mutability
digital cultural record
algorithmic analysis
distant/close, macro/micro, surface/depth
parametrics
cultural mash-ups
algorithm design
data visualization
modelling knowledge
ambient data
collaborative authorship
interdisciplinary teams
use as performance
narrative structures
code as text
software in a cultural context
repurposable content and remix culture
participatory web
read/write/rewrite
meta-medium
polymorphous browsing
Several of the methodologies listed by Gray and Malins (2004) seem to apply to the research presented in this thesis. Exploring, evaluating, analysing, interpreting, synthesising and disseminating research all are part of it. However, looking at the specific methods they collated, the difference becomes clearer as only the following 7 appear relevant (visualization, experimentation with processes, multimedia/hypermedia applications, use of metaphor and analogy, organizational and analytical matrices, curation, and critical writing, publications).
The DH methodologies seem more useful. In terms of design, pata.physics.wtf positions itself in context and the evaluation framework interprets and critiques AMC. Before that I curate the two corpora, digitise them and organise them. Computing comes in at verious stages, to (dis)ambiguate (i.e. pataphysicalise), encode, index, search and match data. The infrastructure is cultural, technical and extensible, relying on the WWW for several spects. Versioning, prototyping and failures all come in during the iterative development process, which involves a lot of experimentation and refactoring. Furthermore, the research methods Burdick et al (2012) list match this project much better (although of course the list above was already only a selection that was deemed relevant; the original list was much larger. See appendix A.3).
3.2 Transdisciplinary
Nicolescu distinguished between 3 different kinds of research “without stable boundaries between the disciplines.”2 (2010).
- Multidisciplinarity
concerns itself with studying a research topic in not just one discipline but in several simultaneously.
- Interdisciplinarity
concerns the transfer of methods from one discipline to another.
- Transdisciplinarity
concerns that which is at once between the disciplines, across the different disciplines, and beyond all disciplines.
The standard epistemological view of science and art is that they are objective and subjective, respectively. So, what does that mean for research conducted between, across and beyond science and art, i.e. research that is transdisciplinary?
Nicolescu criticised the view that science must be objective. He even claimed that any non-scientific knowledge is “cast into the inferno of subjectivity, tolerated at most as a meaningless embellishment or rejected with contempt as a fantasy, an illusion, a regression, or a product of the imagination” (2010). Objectivity, he said, becomes the “supreme criterion of Truth”3
The death of the Subject is the price we pay for objective knowledge.
(Nicolescu 2010)
He went on to quote Werner Heisenberg on the concepts of objective and subjective reality: “we would make a very crude simplification if we want to divide the world in[to] one objective reality and one subjective reality. Many rigidities of the philosophy of the last centuries are born by this black and white view of the world” (Heisenberg, cited in Nicolescu 2010).
The too strong insistence on the difference between scientific knowledge and artistic knowledge comes from the wrong idea that concepts describe perfectly the ‘real things’. […] All true philosophy is situated on the threshold between science and poetry.4
(Heisenberg, cited in Nicolescu 2010)
In transdisciplinarity traditional disciplinary boundaries have no meaning.
Working across disciplines requires a new unique methodology. Nicolescu proposed a methodology of transdisciplinarity as a non-hierarchical ternary partition of ‘Subject, Object and Hidden Third’ (as shown in figure [fig:trans]) rather than the traditional binary partition of ‘Subject versus Object’ (2010).
The old principle “unity in diversity and diversity from unity” is embodied in transdisciplinarity.’
(Nicolescu 2010)
‘unite and conquer’ ↔ ‘divide and conquer’
(Yang 2013)
Hugill and Yang agree that existing research methodologies are unsuitable for transdisciplinary subjects such as Creative Computing (CC). The following is an example of a possible CC research methodology they propose as a starting point (Hugill and Yang 2013):
Review literature across disciplines.
Identify key creative activities.
Analyse the processes of creation.
Propose approaches to support these activities and processes.
Design and implement software following this approach.
Experiment with the resulting system and propose framework.
They go on to propose four standards for CC (Hugill and Yang 2013) namely, (1) resist standardisation, (2) perpetual novelty, (3) continuous user interaction and (4) combinational, exploratory and or transformational.
A different model was suggested by Edmonds and Candy in their Trajectory Model of Practice and Research (TMPR), a framework to “influence practice, inform theory and, in particular, shape evaluation” (2010). Figure 3.2 shows the TMPR which allows for different trajectories between practice, theory and evaluation. Table 3.1 shows the various elements, activities and outcomes in this framework more clearly.
This project positions itself “at once between the disciplines, across the different disciplines, and beyond all disciplines”—making it transdisciplinary. The abolishment of disciplinary boundaries suits the unique context of this research. Pataphysics specifically is highly subjective. Searle highlighted that ontologically subjective topics (such as creativity) can be studied in epistemically objective ways (2015), which, as doctoral research, this project attempts to do.
The Hugill and Yang methodology seems general enough to fit the needs of this project, with all 6 points covered in the various chapters of this thesis.
Review literature across disciplines (chapters 2, 4, 5, 6, and 7).
Define creativity in humans and machines (chapters 4, 5, 6 and 7).
Analyse the relation between the disciplines above (chapter 8).
Propose algorithms to support creativity in machines (chapter 10).
Design and implement software following this approach (chapter 10).
Experiment with the resulting system and propose interpretation/evaluation framework (chapters 12, 13, and 9).
Figure 3.3 shows how the TMPR could be applied to this project.
3.3 Patadisciplinary
So, to summarise, this project draws from several different disciplines as mentioned at the beginning of this chapter: pataphysics—literture, philosophy, art, creativity—cognitive science, AI, DH, and technology—IR, NLP, web development.
- Epistemology
Transdisciplinary, subjective
- Methodology
Creative computing, exploratory, experimental
- Methods
Artefact, literature synthesis, algorithm design, theoretical framework, critical reflection and analysis, rapid incremental prototyping
The general workflow of this project was as follows: (1) critically analyse and synthesise existing literature, (2) develop pataphysical algorithms, (3) design a system to demonstrate algorithms, (4) develop a website as an artefact, (5) define an evaluation and interpretation framework, and (6) analyse results.
As figure 3.3 shows, the practive trajectory of this research is based on the practical development of a website to contain the exploratory search tool and implementation of the theoreical algorithms. The theory trajectory is about defining those algorithms formally in historic and topical context based on a critical survey of related literature. This also includes the development of a theoretical framework for the evaluation and interpretation of creative artefacts. The Evaluation trajectory then is all about the results. That includes an analysis of the work completed. The arrows in the figure indicate how these different trajectories influence each other.