diff --git a/0.00-about-the-authors.rmd b/0.00-about-the-authors.rmd
index 59402f4..bb86fda 100644
--- a/0.00-about-the-authors.rmd
+++ b/0.00-about-the-authors.rmd
@@ -46,9 +46,13 @@ It'll be ok.
We take a pedagogical perspective that focuses on the learning that happens when we make things, when we experiment or otherwise play around with materials and computing, and especially, when/if things break. It's a perspective that finds value in 'failing gloriously', in trying to push ourselves beyond our comfort level. The only thing that you will need therefore to be successful in learning some of the basic issues around digital archaeology is a willingness to consider why things didn't work the way they ought to have, and a web browser. We built this textbook with its very own digital archaeology computer built right in! There's nothing you can break on your own machine, nor does your machine have to be very powerful.
-## How to use this text {-}
+## Students: How to use this text {-}
-Each section in this book is broken down into an overview or discussion of the main concepts, and then followed up with skill-building exercises. The computational environment is running on someone else's servers. When you close the browser, it shuts down. Your work can be saved to your own machine and reloaded into the environment later, or you can 'push' your changes to your own online repository of work. In which case you really ought to work through the sections on [Github & Version Control] and [Open Notebook Research & Scholarly Communication] so you'll be able to get your work and data out of the Jupyter Notebooks and onto space that you control. The best way to use this book is to make sure you have at least one hour blocked out to read through a section, and then two hours to go through the section again if you're working on the exercises. We find that the work goes better if those blocks are interspersed with breaks every twenty minutes or so.
+Each section in this book is broken down into an overview or discussion of the main concepts, and then followed up with skill-building exercises. The computational environment - provided via a Jupyter notebook- is running on someone else's servers. When you close the browser, it shuts down.
+
+**Warning!** The computational notebooks that we provide that are running in the [binder][http://mybinder.org] environment _will time out_ after **ten** minutes' inactivity.
+
+Your work can be saved to your own machine and reloaded into the environment later, or you can 'push' your changes to your own online repository of work (to learn how to push work to a Github repository, see the sections on [Github & Version Control] and [Open Notebook Research & Scholarly Communication] so you'll be able to get your work and data out of the Jupyter Notebooks and onto space that you control). The best way to use this book is to make sure you have at least one hour blocked out to read through a section, and then two hours to go through the section again if you're working on the exercises. We find that the work goes better if those blocks are interspersed with breaks every twenty minutes or so.
Do you notice that stripe down the side of the screen at the right? That's a tool-bar for annotating the text, using a tool called `Hypothes.is`. If you highlight any text (go ahead, highlight that phrase right now by right-clicking and dragging your mouse!) a little pop-up will ask you if you want to annotate or highlight the text. If you choose annotate, a writing pane will open on the right. Using the Hypothesis tool requires a reader to create a login and account with Hypothesis, which is managed by the Hypothesis site, not us.
diff --git a/01-introduction.rmd b/01-introduction.rmd
index 2c99760..a8da9d8 100644
--- a/01-introduction.rmd
+++ b/01-introduction.rmd
@@ -2,8 +2,10 @@
Digital archaeology as a field rests upon the creative use of primarily open-source and/or open-access materials to archive, reuse, visualize, analyze and communicate archaeological data. This reliance on open-source and open-access is a political stance that emerges in opposition to archaeology’s past complicity in colonial enterprises and scholarship; digital archaeology resists the digital neo-colonialism of Google, Facebook, and similar tech giants that typically promote disciplinary silos and closed data repositories. Specifically, digital archaeology encourages innovative, reflective, and critical use of open access data and the development of digital tools that facilitate linkages and analysis across varied digital sources.
-To that end, this document you are reading is integrated with a cloud-based digital exploratory laboratory of multiple cloud-computing tools with teaching materials that instructors will be able to use 'out-of-the-box' with a single click, or to remix as circumstances dictate. Part of our inspiration comes from the ‘DHBox’ project from CUNY (City University of New York, [(link)](http://dhbox.org), a project that is creating a ‘digital humanities laboratory’ in the cloud. While the tools of the digital humanities are congruent with those of digital archaeology, they are typically configured to work with texts rather than material culture in which archaeologists specialise. The second inspiration is the open-access guide ‘The Programming Historian’, which is a series of how-tos and tutorials [(link)](http://programminghistorian.org) pitched at historians confronting digital sources for the first time. A key challenge scholars face in carrying out novel digital analysis is how to install or configure software; each 'Programming Historian' tutorial therefore explains in length and in detail how to configure software. The present e-textbook merges the best of both approaches to create a singular experience for instructors and students: a one-click digital laboratory approach, where installation of materials is not an issue, and with carefully designed tutorials and lessons on theory and practice in digital archaeology.
+To that end, this document you are reading is integrated with live open code notebooks that can be re-used, altered, or extended. Part of our inspiration comes from the ‘DHBox’ project from CUNY (City University of New York, [(link)](http://dhbox.org), a project that is creating a ‘digital humanities laboratory’ in the cloud. While the tools of the digital humanities are congruent with those of digital archaeology, they are typically configured to work with texts rather than material culture in which archaeologists specialise. The second inspiration is the open-access guide ‘The Programming Historian’, which is a series of how-tos and tutorials [(link)](http://programminghistorian.org) pitched at historians confronting digital sources for the first time. A key challenge scholars face in carrying out novel digital analysis is how to install or configure software; each 'Programming Historian' tutorial therefore explains in length and in detail how to configure software. The present e-textbook merges the best of both approaches to create a singular experience for instructors and students: a one-click digital laboratory approach, where installation of materials is not an issue, and with carefully designed tutorials and lessons on theory and practice in digital archaeology.
-This is not a textbook about learning how to code. Rather, it is about instilling the habits of thought that will enable success when confronted with digital novelty, the habits of thought that will enable you to determine how to work with digital materials, and the habits of thought that permit you to see where and when digital approaches will make the difference in your research. Skills change; techniques evolve; new tools emerge. Habits of thought are hard to cultivate but have staying power!
+This is not a textbook about learning how to code. Rather, it is about instilling the habits of thought that will enable success when confronted with digital novelty, the habits of thought that will enable you to determine how to work with digital materials, and the habits of thought that permit you to see where and when digital approaches will make the difference in your research. Skills change; techniques evolve; new tools emerge. Habits of thought are hard to cultivate but have staying power!
-Through this textbook, we aim to offer a learners'-perspective-view on digital methods in archaeology, that is, how we might think with, and through, digital sources of information, digital tools and technologies and their relationship with society. We are deeply aware of how rapidly both digital sources and technologies can change, particularly on the Web; we therefore present this e-textbook and open-learning environment as a guide to best practices when working with available digital data and digital tools, what kinds of analysis are possible, how to perform these analytical techniques, and how you might publish your data, making them re-usable for another scholar and ethics and ethical issues in doing digital archaeology.
+Through this textbook, we aim to offer a learners'-perspective-view on digital methods in archaeology, that is, how we might think with, and through, digital sources of information, digital tools and technologies and their relationship with society. We are deeply aware of how rapidly both digital sources and technologies can change, particularly on the Web; we therefore present this e-textbook and open-learning environment as a guide to best practices when working with available digital data and digital tools, what kinds of analysis are possible, how to perform these analytical techniques, and how you might publish your data, making them re-usable for another scholar and ethics and ethical issues in doing digital archaeology.
+
+We have not elected to try to _cover_ every possible topic. By design, this book is meant to grow, branch, and change with time. It is meant to foster _uncoverage_ and be used to supplement or complement an instructor's own situated approach. Annotate the text using the [Hypothes.is](http://hypothes.is). Take it to bits. Use and adapt the parts that make most sense in your own particular learning context.
diff --git a/01.1-whatisdigitalarchaeology.Rmd b/01.1-whatisdigitalarchaeology.Rmd
index f4c221d..657b92c 100644
--- a/01.1-whatisdigitalarchaeology.Rmd
+++ b/01.1-whatisdigitalarchaeology.Rmd
@@ -6,11 +6,11 @@ Our approach in this volume is to resolve that seeming paradox by providing not
Digital archaeology is necessarily a public archaeology. This is its principal difference with what has come before, for never forget, there has been at least a half-century of innovative use of computational power for archaeological knowledge building.
-Geospatial, digital and Web-based tools are now central to carrying out archaeological research and to communicating archaeological information in a globalized world. Until recently, the accumulation and effective management of digital archaeological data have been the primary focus of archaeologists (Evans and Daly 2006). Under this model, scholars emphasize the ‘integration’ into archaeology of computing technologies, and how, by utilizing current available computing memory and processer speed, one does archaeology, only better (Daly and Evans 2006: 1). This situation in turn demonstrates the ‘marriage between the two’, archaeology and computing (Daly and Evans 2006: 2).
+Geospatial, digital and Web-based tools are now central to carrying out archaeological research and to communicating archaeological information in a globalized world. Until recently, the accumulation and effective management of digital archaeological data have been the primary focus of archaeologists (Evans and Daly 2006). Under this model, scholars emphasize the ‘integration’ into archaeology of computing technologies, and how, by utilizing current available computing memory and processer speed, one does archaeology, only better (Daly and Evans 2006: 1). This situation in turn demonstrates the ‘marriage between the two’, archaeology and computing (Daly and Evans 2006: 2).
For Evans and Daly (2006), writing in the first decade of the 21st century, digital archaeology was synonymous with the use of Information and Communication Technology or ICT, and reflected wider efforts at that moment to transform education through newly available digital tools. Some scholars and policy makers believed that digital technologies were the answer to pressing global social issues such as poverty, a point that we will discuss later.
-More recently, in his inaugural editorial for the open-access journal, Frontiers in Digital Humanities, [@Costopoulos_2016] argues that ‘digital archaeology has been [here] a while’. Computing in archaeology, that is ‘doing archaeology digitally’ as Costopolous remarks, constitutes a ‘state of normality’ in archaeological practice. This view places emphasis on the availability of digital tools and their use in institutional contexts, overlooking the highly structured nature of social groups that employ these tools, and where, how and why these technologies are created and used. While fruitful, these views tend to obscure broader developments in the social sciences and humanities, of which archaeology is a part, and underestimate the changing relationship between archaeology and society.
+More recently, in his inaugural editorial for the open-access journal, Frontiers in Digital Humanities, @Costopoulos_2016 argues that ‘digital archaeology has been [here] a while’. Computing in archaeology, that is ‘doing archaeology digitally’ as Costopolous remarks, constitutes a ‘state of normality’ in archaeological practice. This view places emphasis on the availability of digital tools and their use in institutional contexts, overlooking the highly structured nature of social groups that employ these tools, and where, how and why these technologies are created and used. While fruitful, these views tend to obscure broader developments in the social sciences and humanities, of which archaeology is a part, and underestimate the changing relationship between archaeology and society.
### A distant view
Ethan Watrall has drawn the history of computational archaeology/digital archaeology all the way back to the pioneering work of James Deetz in the 1960s, who used computers at MIT to perform stylistic analyses of Arikara ceramics [@watrall_archaeology_2017, @deetz_dynamics_1965]. Most early interest in computation for archaeology was centred on the potential for computational databases, although ambition often out-stripped capability. By the 1970s, serious efforts were being put into work to build the infrastructural knowledge necessary to make and usefully query archaeological datasets. One can see this concern play out by considering a `topic model` [@shawn_graham_digital_2014] of the early volumes of the Computer Applications in Archaeology (a topic model is a way of deducing latent patterns of discourse within text, based on patternings of words [See @graham_getting_2012]):
@@ -21,13 +21,13 @@ topic 6: but, they, one, time, their, all, some, only, will, there, would, what,
topic 20: some, will, many, there, field, problems, may, but, archaeologists, excavation, their, they, recording, however, record, new, systems, most, should, need
-The beginnings of the CAA are marked by hesitation and prognostication: what *are* computers for, in archaeology? There is a sense that for archaeologists, computation is something that will be useful insofar as it can be helpful for recording information in the field. By the 1980s desktop computing was becoming sufficiently widespread that the use of geographic information systems was feasible for more and more archaeologists. The other 'killer app' of the time was computer-aided design, which allowed metric 3d reconstructions from the plans drawn on site by excavators. Yet, computational resources were still limited enough that computing was not something that one could merely 'play' with. Software was costly, computation took time, and training resources were put into learning the proprietary packages that existed (rather than coding knowledge). By the 1990s, the introduction of the cd-rom and the shift in PC gaming technologies from primarily text-based to graphical based games led to teaching simulations for archaeology, most notably T. Douglas Price and Anne Birgitte Gebauer's _Adventures in Fugawiland_. Watrall identifies the emergence of the web as being not so much a boon for computational archaeology as it was for public archaeology (although the pioneering journal _Internet Archaeology_ was first published in 1996); nevertheless, the birth of the web (which it must be remembered is _distinct from_ and _overlays_ the internet) allowed for a step-change in the effectiveness of the dissemination of open-source software and code, including practices for remote collaboration on code that are now beginning to percolate into scholarly publication.
+The beginnings of the CAA are marked by hesitation and prognostication: what *are* computers for, in archaeology? There is a sense that for archaeologists, computation is something that will be useful insofar as it can be helpful for recording information in the field. By the 1980s desktop computing was becoming sufficiently widespread that the use of geographic information systems was feasible for more and more archaeologists. The other 'killer app' of the time was computer-aided design, which allowed metric 3d reconstructions from the plans drawn on site by excavators. Yet, computational resources were still limited enough that computing was not something that one could merely 'play' with. Software was costly, computation took time, and training resources were put into learning the proprietary packages that existed (rather than coding knowledge). By the 1990s, the introduction of the cd-rom and the shift in PC gaming technologies from primarily text-based to graphical based games led to teaching simulations for archaeology, most notably T. Douglas Price and Anne Birgitte Gebauer's _Adventures in Fugawiland_. Watrall identifies the emergence of the web as being not so much a boon for computational archaeology as it was for public archaeology (although the pioneering journal _Internet Archaeology_ was first published in 1996); nevertheless, the birth of the web (which it must be remembered is _distinct from_ and _overlays_ the internet) allowed for a step-change in the effectiveness of the dissemination of open-source software and code, including practices for remote collaboration on code that are now beginning to percolate into scholarly publication.
The 2000s have seen, insofar as digital archaeology is concerned, a replay of the earlier episodes of computational archaeology, concommitant with each subsequent web 'revolution' (ie, so-called web 2.0, web 3.0 etc). Works such as [@evans_digital_2006] and [@kansa_archaeology_2011] are broadly concerned more with questions of infrastructure and training, while the more recent _Mobilizing the Past_ deal with problems of training, and the ethical issues that the emerging digital surveillance permitted by our networked society presents to the practice of archaeology (and public archaeology). Perhaps the most promising new digital technologies to emerge in recent years include methods for linking open archaeological data via the web (ie, freeing various 'silos' of disciplinary knowledge so that the semantic connections between them can be followed and queried) and various mixed-reality approaches (virtual reality, augmented reality, 3d printing, and the so-called internet of things or the practice of wiring everything that can be wired to the web). The 2000s have also seen a growing realization that our digital tools and their algorithmic biases not only permit interesting questions to be asked about the past, but also inhibit points of view or impose their own worldviews upon the past in ways that may damage communities and/or scholarship. This reflective critique of computation in the service of archaeology marks digital archaeology within the ambit of the digital humanities (despite the division between anthropological and humanistic archaeologies).
### Is digital archaeology part of the digital humanities?
-In recent years - certainly the last decade - an idea called 'the digital humanities' has been percolating around the academy. It is a successor idea to 'humanities computing', but it captures that same distinction between discussed above. Digital archaeology has developed alongside the digital humanities, sometimes intersecting with it (notably, there was a major archaeological session at the annualinternational Alliance of Digital Humanities Organizations ([ADHO](http://adho.org/)) DH conference in 2013).
+In recent years - certainly the last decade - an idea called 'the digital humanities' has been percolating around the academy. It is a successor idea to 'humanities computing', but it captures that same distinction between discussed above. Digital archaeology has developed alongside the digital humanities, sometimes intersecting with it (notably, there was a major archaeological session at the annualinternational Alliance of Digital Humanities Organizations ([ADHO](http://adho.org/)) DH conference in 2013).
The various component organizations of the ADHO have been meeting in one form or another since the 1970s; so too the Computer Applications in Archaeology Conference has been publishing its proceedings since 1973. Archaeologists have been running simulations, doing spatial analysis, clustering, imaging, geophysicing, 3d modeling, neutron activation analyzing, x-tent modeling , etc, for what seems like ages. Happily, there is no one definition of 'dh' that everyone agrees on (see the various definitions collected at [http://definingdh.org/](http://definingdh.org/); reload the page to get a new definition). For us, a defining _characteristic_ of DH work is that public use we discussed above. But, another characteristic that we find useful to consider is the _purpose_ to which computation is put in DH work. This means that digital work also has to be situated in the contexts of power and access and control (which sometimes means that digital work is mis-characterised as being part of a 'neo-liberal' agenda to reduce knowledge work to base profit motifs, eg Brouiellet; more thoughtful work about the confluence of the digital with neoliberalism may be found in Caraher xxxx and Kansa xxxx and Greenspan xxx. We discuss the ethical dimensions to digital work more fully in [The Ethics of Big Data in Archaeology].)
@@ -60,7 +60,7 @@ To get that more humane hack that Liu seeks, Liu suggests that the historical de
In which case, the emergence of digital archaeologists and historians in the last decade might be the loci of the humane hacks – if we move into that space where we engage the arts. Indeed, the seminal anthropologist Tim Ingold makes this very argument with reference to his own arc as a scholar, 'From Science to Art and Back Again':
-> Revisiting science and art: which is more ecological now? Why is art leading the way in promoting radical ecological awareness? The goals of today's science are modelling, prediction and control. Is that why we turn to art to rediscover the humility that science has lost?
+> Revisiting science and art: which is more ecological now? Why is art leading the way in promoting radical ecological awareness? The goals of today's science are modelling, prediction and control. Is that why we turn to art to rediscover the humility that science has lost?
We need to be making art. Digital archaeology naturally pushes in that direction.
@@ -71,7 +71,7 @@ We need to be making art. Digital archaeology naturally pushes in that direction
+ In that deformative practice, it is in some ways extremely aligned with artistic ways of knowing
+ Digital archaeology is part of the digital humanities, and in many ways, presaged current debates and trends in that field.
-All of these aspects of digital archaeology exist along a continuum. In the remainder of this chapter, we give you a 'boot-camp' to get you to the point where you can begin to wonder about deformation and the public entanglement with your work.
+All of these aspects of digital archaeology exist along a continuum. In the remainder of this chapter, we give you a 'boot-camp' to get you to the point where you can begin to wonder about deformation and the public entanglement with your work.
### Exercises
@@ -83,13 +83,13 @@ The first steps in going digital are quite easy. They are fundamentally a questi
Have you ever fought with Word or another wordprocessor, trying to get things just right? Word processing is a mess. It conflates writing with typesetting and layout. Sometimes, you just want to get the words out. Othertimes, you want to make your writing as accessible as possible... but your intended recipient can't open your file, because they don't use the same wordprocessor. Or perhaps you wrote up some great notes that you'd love to have in a slideshow; but you can't, because copying and pasting preserves a whole lot of extra gunk that messes up your materials. Similarly, while many archaeologists will use Microsoft Excel to manipulate tabular data (artifact measurements, geochemistry data, and so on), Excel is well known for both [corrupting data](https://github.com/jennybc/scary-excel-stories) and for being impossible to replicate (ie, the series of clicks to manipulate or perform an analysis differ depending on the individual's particular installation of Excel).
-The answer is to separate your content from your tool, and your analytical processes separate from your data. This can help keep your thinking clear, but it also has a more nuts-and-bolts practical dimension. *A computer will always be able to read a text file*. That is to say: you've futureproofed your material. Any researcher will have old physical discs or disc drives or obsolete computers lying around. It is not uncommon for a colleague to remark, 'I wrote this in Wordperfect and I can't open this any more'. Graham's MA thesis is trapped on a 3.5" disc drive that was compressed using a now-obsolete algorithm and it cannot be recovered. If, on the other hand, he had written the text as a .txt file, and saved the data as .csv tables, those materials would *continue* to be accessible. If the way you have manipulated or cleaned the data is written out as a `script`, then a subsequent investigator (or even your future self) can re-run the exact sequence of analysis, or re-write the script into the equivalent steps in another analytical language.
+The answer is to separate your content from your tool, and your analytical processes separate from your data. This can help keep your thinking clear, but it also has a more nuts-and-bolts practical dimension. *A computer will always be able to read a text file*. That is to say: you've futureproofed your material. Any researcher will have old physical discs or disc drives or obsolete computers lying around. It is not uncommon for a colleague to remark, 'I wrote this in Wordperfect and I can't open this any more'. Graham's MA thesis is trapped on a 3.5" disc drive that was compressed using a now-obsolete algorithm and it cannot be recovered. If, on the other hand, he had written the text as a .txt file, and saved the data as .csv tables, those materials would *continue* to be accessible. If the way you have manipulated or cleaned the data is written out as a `script`, then a subsequent investigator (or even your future self) can re-run the exact sequence of analysis, or re-write the script into the equivalent steps in another analytical language.
A .txt file is simply a text file; a .csv is a text file that uses commas to separate the text into columns. Similarly, a .md file is a text file that uses things like `#` to indicate headers, and `_` to show where italicized text starts and stops. A script, in a play, tells you what to say and do. A `script` for a language like `R` or `Python` does the same thing for the computer, and has the advantage that it is human-readable and annotatable as well, because its format *is still a simple text file*. Scripts you might encounter could have the `.r` or `.py` or `.sh` file extensions. You can open these in a text editor and see what the computer is being instructed to do. Annotations or comments in the script can be set off in various ways, and help the researcher know what is happening or is intended to happen at various points. Let's begin by creating some simple text files to document our research process, in the Markdown format.
-1. A nice place to practice writing in markdown that shows you immediately how your text might be rendered when turned into html, pdf, or Word doc is [Dillinger.io](http://dillinger.io). Go there now to try it out. Write a short piece on why you're interested in Digital Archaeology.
-a. Include a blockquote from the introduction to this book.
-b. Include two links to an external site.
+1. A nice place to practice writing in markdown that shows you immediately how your text might be rendered when turned into html, pdf, or Word doc is [Dillinger.io](http://dillinger.io). Go there now to try it out. Write a short piece on why you're interested in Digital Archaeology.
+a. Include a blockquote from the introduction to this book.
+b. Include two links to an external site.
c. Embed an image.
Here are two short demonstration videos:
@@ -117,11 +117,10 @@ Notice, when you're on your repository's page, that there is a button to 'create
4. Click on the green 'commit' button at the bottom of the page when you're done.
-Now we'll upload a file into your repository.
+Now we'll upload a file into your repository.
5. Find the markdown file on your machine that you created with Dillinger. You can drag-and-drop this onto the list of files in your repository; Github will know that you want to upload it. **OR** you can click on the 'upload files' button, and then 'select files' and click on the file that way (much like adding an attachment to an email).
-6. Github will upload the file; you can upload more files at this point, or click on the green 'commit' button at the bottom.
-
-** As you work through this book, we encourage you to write your thoughts, observations, or results in simple text files.** This is good practice whether or not you embark on a full-blown digital project, because ultimately, if you use a computer in your research, you *have* gone digital.
+6. Github will upload the file; you can upload more files at this point, or click on the green 'commit' button at the bottom.
+**As you work through this book, we encourage you to write your thoughts, observations, or results in simple text files.** This is good practice whether or not you embark on a full-blown digital project, because ultimately, if you use a computer in your research, you *have* gone digital.
diff --git a/01.5-failingproductively.Rmd b/01.5-failingproductively.Rmd
index 5d6cbe2..242e7b4 100644
--- a/01.5-failingproductively.Rmd
+++ b/01.5-failingproductively.Rmd
@@ -1,13 +1,10 @@
## Failing Productively
-We have found that students are very nervous about doing digital work because, 'what if it breaks?' and 'what if I can't get it to work?' This is perhaps a result of high-stakes testing and the ways we as educators have inculcated all-or-nothing grading in our courses. There is no room for experimentation, no room for trying things out when the final essay is worth 50% of the course grade, for instance. Playing it safe is a valid response in such an environment. A better approach, from a pedagogical point of view, is to encourage students to explore and try things out, with the grading being focused on documenting the _process_ rather than on the final outcome. We will point the reader to Daniel Paul O'Donnel's concept of the [unessay; more details behind the link](http://people.uleth.ca/~daniel.odonnell/Teaching/the-unessay).
+We have found that students are very nervous about doing digital work because, 'what if it breaks?' and 'what if I can't get it to work?' This is perhaps a result of high-stakes testing and the ways we as educators have inculcated all-or-nothing grading in our courses. There is no room for experimentation, no room for trying things out when the final essay is worth 50% of the course grade, for instance. Playing it safe is a valid response in such an environment. A better approach, from a pedagogical point of view, is to encourage students to explore and try things out, with the grading being focused on documenting the _process_ rather than on the final outcome. We will point the reader to Daniel Paul O'Donnel's concept of the [unessay; more details behind the link](http://people.uleth.ca/~daniel.odonnell/Teaching/the-unessay).
-Our emphasis on open notebooks has an ulterior motive, and that is to surface the many ways in which digital work sometimes fails. We want to introduce to you the idea of 'failing productively' because there is such a thing as an unproductive failure. There are ways of failing that do not do us much good, and we need - especially with digital work - to consider what 'fail' actually can mean. In the technology world, there are various slogans surrounding the idea of 'fail' - fail fast; move fast and break things; fail better; for instance.
-
-When we talk about 'failing productively' or failing better, it is easy for critics of digital archaeology (or the digital humanities; see Allington et al 2016 but contra: Greenspan 2016) to connect digital work to the worst excesses of the tech sector. But again, this is to misunderstand what 'fail' should mean. The understanding of many tech startup folks that valorizing failure as a license to burn through funding has caused a lot of harm. The tech sector failed to understand the humanistic implication of the phrase, and instead took it literally to mean 'a lack of success is itself the goal'. But where does it come from?
-
-The earliest use of the 'fail better' idea that we have found outside the world of literature and criticism seems to occur during the first tech boom, where it turns up in everything from diet books to learning to play jazz, to technology is in _The Quest for a Unified Theory of Information_. CITATION (Wolfgang Hofkirchner) That book according to Google Scholar at the time of writing has been cited 13 times, but the things that cite it have themselves been cited over 600 times. We are here merely speculating on where this mantra of the fast fail, fail better, comes from and how it spreads, but it would be a very interesting topic to explore.
+Our emphasis on open notebooks has an ulterior motive, and that is to surface the many ways in which digital work sometimes fails. We want to introduce to you the idea of 'failing productively' because there is such a thing as an unproductive failure. There are ways of failing that do not do us much good, and we need - especially with digital work - to consider what 'fail' actually can mean. In the technology world, there are various slogans surrounding the idea of 'fail' - fail fast; move fast and break things; fail better; for instance.
+When we talk about 'failing productively' or failing better, it is easy for critics of digital archaeology (or the digital humanities; see Allington et al 2016 but contra: Greenspan 2016) to connect digital work to the worst excesses of the tech sector. But again, this is to misunderstand what 'fail' should mean. The understanding of many tech startup folks that valorizing failure as a license to burn through funding has caused a lot of harm. The tech sector failed to understand the humanistic implication of the phrase, and instead took it literally to mean 'a lack of success is itself the goal'.
Perhaps a better understanding of what 'fail' should mean is as something akin to what Nassim Taleb called 'antifragility'. The fragile thing breaks under stress and randomness; the resilient thing stays the same; and the anti-fragile thing actually gets stronger as it is exposed to randomness. Kids' bones for instance need to be exposed to shocks in order to get stronger. Academia's systems are 'fragile' in that they do not tolerate fail; they are to a degree resilient, but they are not 'antifragile' in Taleb's sense. The idea that 'fail' can break that which is 'fragile' is part of the issue here. So silicon valley really means 'fail' in the sense of 'antifragile' but they frequently forget that; academia sees 'fail' as the breaking of something fragile; and so the two are at loggerheads. Indeed, the rhetorical moves of academe often frame weak results - fails - as actual successes, thus making the scholarship fragile (hence the fierceness of academic disputes when results are challenged, sometimes). To make scholarship anti-fragile - to extract the full value of a fail and make it be productive, we need remember only one thing:
**A failure shared is not a failure.**
@@ -29,7 +26,7 @@ There are fails, and then there are fails. Croxall and Warnick identify a taxono
5. Failing to Share
-The first is the simplest: something simply did not work. The code is buggy, dust and grit got into the fan and the hardware seized. The second, while labeled 'human failure' really means that the _context_, the framework for encountering the technology was not erected properly, leading to a failure to appreciate what the technology could do or how it was intended to be used. This kind of failure can also emerge when we ourselves are not open to the possibilities or work that the technology entails. The next two kinds of failure emerge from the first in that they are ways of dealing with the first two kinds of failure. 'Failure as Artifact' means that we seek out examples of failures as things to study, working out the implications of why something did not work. Finally, 'Failure as Epistemology' purposely builds the opportunity to fail into the research design, such that each succeeding fail (of type 1 or type 2) moves us closer to the solution that we need. The first two refer to what happened; the second two refer to our response and how we react to the first two (*if* we react at all). The key to productive failure as we envision it is to recognize when one's work is suffering from a type 1 or type 2 fail, and to transform it to a type 3 or type 4. Perhaps there should be a fifth category though, a failure to share. For digital archaeology to move forward, we need to know where the fails are and how to move beyond them, such that we move forward as a whole. Report not just the things that work, but also the fails. That is why we keep open research notebooks.
+The first is the simplest: something simply did not work. The code is buggy, dust and grit got into the fan and the hardware seized. The second, while labeled 'human failure' really means that the _context_, the framework for encountering the technology was not erected properly, leading to a failure to appreciate what the technology could do or how it was intended to be used. This kind of failure can also emerge when we ourselves are not open to the possibilities or work that the technology entails. The next two kinds of failure emerge from the first in that they are ways of dealing with the first two kinds of failure. 'Failure as Artifact' means that we seek out examples of failures as things to study, working out the implications of why something did not work. Finally, 'Failure as Epistemology' purposely builds the opportunity to fail into the research design, such that each succeeding fail (of type 1 or type 2) moves us closer to the solution that we need. The first two refer to what happened; the second two refer to our response and how we react to the first two (*if* we react at all). The key to productive failure as we envision it is to recognize when one's work is suffering from a type 1 or type 2 fail, and to transform it to a type 3 or type 4. Perhaps there should be a fifth category though, a failure to share. For digital archaeology to move forward, we need to know where the fails are and how to move beyond them, such that we move forward as a whole. Report not just the things that work, but also the fails. That is why we keep open research notebooks.
Lets consider some digital projects that we have been involved in, and categorize the kinds of fails they suffered from. We turn first to the HeritageCrowd project that Graham established in 2011. This project straddled community history and cultural history in a region poorly served by the internet. It was meant to crowd-source intangible heritage via a combination of web-platform and telephony (people could phone in with stories, which were automatically transcribed and added to a webmap). The first write-up of the project was published just as the project started to get underway (CITATION GRAHAM ETAL). It's what happened next that is of interest here.
@@ -42,13 +39,12 @@ Why did it fail? It was a combination of at least four distinct problems (GRAHAM
3. Graham ignored security warnings from the main platform's maintainers
4. Backups and versioning: there were none.
-Graham's fails here are of both type 1 and type 2. In terms of type 2, his failure to keep careful notes on how the various pieces of the project were made to fit together meant that he lacked the framework to understand how he had made the project vulnerable to attack. The actual fail point of the attack - that's a type 1 fail, but could have been avoided if Graham had participated more in the spirit of open software development, eg, read the security warnings in the developer forum! When Graham realized what had happened to his project, he was faced with two options. One option, having already published a piece on the project that hailed its successes and broad lessons for crowdsourcing cultural heritage, would have been to quietly walked away from the project (perhaps putting up a new website avverring that version 2.0 was coming, pending funding). The other option was to warn folks to beef up the security and backups for their own projects. At the time, crowdsourcing was very much an academic fashion and Graham opted for the second option in that spirit. In doing this, the HeritageCrowd project became a fail of type 3, an artifact for study and reflection. The act of blogging his post-mortem makes this project also an instance of type 5, or the communication of the fail. It is worth pointing out here that the _public_ sharing of this failure is not without issues. As we indicated in the [Open Notebook Research & Scholarly Communication] section, the venue for sharing what hasn't worked and the lessons learned is highly contingent on many factors. Graham, as a white male tenure-track academic on the web in 2012, could share openly that things had not worked. As the web has developed in the intervening years, and with the increasing polarization of web discourse into broad ideological camps, it may well not be safe for someone in a more precarious position to share so openly. One must keep in mind one's own situation and adapt what we argue for here accordingly. Sharing fails can be done with close colleagues, students, specialist forums and so on.
+Graham's fails here are of both type 1 and type 2. In terms of type 2, his failure to keep careful notes on how the various pieces of the project were made to fit together meant that he lacked the framework to understand how he had made the project vulnerable to attack. The actual fail point of the attack - that's a type 1 fail, but could have been avoided if Graham had participated more in the spirit of open software development, eg, read the security warnings in the developer forum! When Graham realized what had happened to his project, he was faced with two options. One option, having already published a piece on the project that hailed its successes and broad lessons for crowdsourcing cultural heritage, would have been to quietly walked away from the project (perhaps putting up a new website averring that version 2.0 was coming, pending funding). The other option was to warn folks to beef up the security and backups for their own projects. At the time, crowdsourcing was very much an academic fashion and Graham opted for the second option in that spirit. In doing this, the HeritageCrowd project became a fail of type 3, an artifact for study and reflection. The act of blogging his post-mortem makes this project also an instance of type 5, or the communication of the fail. It is worth pointing out here that the _public_ sharing of this failure is not without issues. As we indicated in the [Open Notebook Research & Scholarly Communication] section, the venue for sharing what hasn't worked and the lessons learned is highly contingent on many factors. Graham, as a white male tenure-track academic on the web in 2012, could share openly that things had not worked. As the web has developed in the intervening years, and with the increasing polarization of web discourse into broad ideological camps, it may well not be safe for someone in a more precarious position to share so openly. One must keep in mind one's own situation and adapt what we argue for here accordingly. Sharing fails can be done with close colleagues, students, specialist forums and so on.
-If we are successful with ODATE, and the ideas of productive fail begin to permeate more widely in the teaching of digital archaeology, then a pedagogy that values fail will with time normalize such 'negative results'. We are motivated by this belief that digital archaeology is defined by the productive, pedagogical fail. It is this aspect of digital archaeology that also makes it a kind of public archaeology, and that failing in public can be the most powerful thing that digital archaeology offers the wider field.
+If we are successful with ODATE, and the ideas of productive fail begin to permeate more widely in the teaching of digital archaeology, then a pedagogy that values fail will with time normalize such 'negative results'. We are motivated by this belief that digital archaeology is defined by the productive, pedagogical fail. It is this aspect of digital archaeology that also makes it a kind of public archaeology, and that failing in public can be the most powerful thing that digital archaeology offers the wider field.
-We implore you to do your research so that others can retrace your steps; even a partial step forward is a step forward! When you find a colleague struggling, give positive and meaningful feedback. Be open about your own struggles, but get validation of your skills if necessary. Build things that make you feel good about your work into your work.
+We implore you to do your research so that others can retrace your steps; even a partial step forward is a step forward! When you find a colleague struggling, give positive and meaningful feedback. Be open about your own struggles, but get validation of your skills if necessary. Build things that make you feel good about your work into your work.
### Exercises
What are the nature of your own fails? Reflect on a ‘fail’ that happened this past year. Where might it fit on the taxonomy? Share this fail via your open notebook, blog, or similar, with your classmates. How can your classmate convert their fails into types three or four?
-
diff --git a/02.2-cleaning-data-with-open-refine.rmd b/02.2-cleaning-data-with-open-refine.rmd
deleted file mode 100644
index 6117b12..0000000
--- a/02.2-cleaning-data-with-open-refine.rmd
+++ /dev/null
@@ -1,7 +0,0 @@
-## Cleaning Data with Open Refine
-
-blahde blah blah
-
-### discussion
-
-### exercises
\ No newline at end of file
diff --git a/02.2.1-databases.Rmd b/02.2-databases.rmd
similarity index 97%
rename from 02.2.1-databases.Rmd
rename to 02.2-databases.rmd
index 3933238..fe9606b 100644
--- a/02.2.1-databases.Rmd
+++ b/02.2-databases.rmd
@@ -20,3 +20,6 @@ http://www.datacarpentry.org/spreadsheets-socialsci/
SQL for Social Science Data
http://www.datacarpentry.org/sql-socialsci/
+
+Cleaning Data with Open Refine
+
diff --git a/02.4-introdigitallibraries.Rmd b/02.4-introdigitallibraries.Rmd
index 790ac5c..87b8455 100644
--- a/02.4-introdigitallibraries.Rmd
+++ b/02.4-introdigitallibraries.Rmd
@@ -2,4 +2,6 @@
yadda linked open data in here?
-- there should be a discussion here on the shape of data; not everything comes in flat files (explain what that means); also json, geojson,
\ No newline at end of file
+- there should be a discussion here on the shape of data; not everything comes in flat files (explain what that means); also json, geojson,
+
+SG June 18: maybe here's where the archdata, finds.org.uk notebooks could go?
\ No newline at end of file
diff --git a/02.5-commandlineapis.Rmd b/02.5-commandlineapis.Rmd
index d3e422d..9a014e8 100644
--- a/02.5-commandlineapis.Rmd
+++ b/02.5-commandlineapis.Rmd
@@ -1,6 +1,14 @@
## Command Line Methods for Working with APIs
+[todo]
++ simple api call to chronicling america
++ more complex to open context
++ some simple plots in the data from open context
++ calling leaflet from notebook, mapping open context <- but maybe do this in 3.4
+
+see jupyter notebooks, https://github.com/o-date/open-context-jupyter
+
### Working with Omeka
yadda
diff --git a/04.1-3d-photogrammetry.rmd b/04.1-3d-photogrammetry.rmd
index 5b03a00..32c7dca 100644
--- a/04.1-3d-photogrammetry.rmd
+++ b/04.1-3d-photogrammetry.rmd
@@ -7,7 +7,7 @@ Laser scanning, on the other hand, involves shooting rays of light onto an objec
In this chapter, we'll cover some of the basic principles of how sfm works while pointing you to more detailed discussions. We also provide links in the 'further readings' to some recent case studies using 3d photogrammetry in novel ways.
-Basic principles
+### Basic principles
- image capture
- image matching
@@ -50,69 +50,94 @@ then what?
- download to your computer; clean up
-### discussion
+from regard3d:
+To obtain a 3D model, the following steps are performed:
+
+For each image, features (sometimes also called keypoints) are detected. Features are points in an object that have a high probability to be found in different images of the same object, for example corners, edges etc. Regard3D uses A-KAZE for this purpose.
+For each feature, a mathematical descriptor is calculated. This descriptor has the characteristic that descriptors of the same point in an object in different images (seen from different viewpoints) is similar. Regard3D uses LIOP (Local Intensity Order Pattern) for this purpose.
+The descriptors from different images are matched and geometrically filtered. The result of this step is a collection of matches between each image pair.
+Now "tracks" are calculated. For each feature that is part of a match in an image pair, it is searched also in other images. A track is generated from features if these features satisfy some conditions, for example a track is seen in at least 3 images.
+The next step is the triangulation phase. All the matches of all the image pairs are used to calculate:
+The 3D position and characteristic of the "camera", i.e. where each image was shot and the visual characteristics of the camera
+The 3D position of each "track" is calculated
+The result of the triangulation phase is a sparse point cloud. In order to obtain a more dense point cloud ("densification"), several algorithms can be used.
+The last step is called "Surface generation". The point clouds are used to generate a surface, either with colored vertices or with a texture.
+
+
### exercises
-### nuts and bolts
-In order to use something like [Regard3d](http://www.regard3d.org/) to build models from _pictures you've taken with your cellphone_, you need to add certain information to
+While there are command-line applications (like [VSFM](http://ccwu.me/vsfm/)) for photogrammetry, running such things from a Jupyter notebook does not work very well. VSFM *can* be installed in something like DHBox (but it's not for the faint-of-heart, see eg [this](http://www.10flow.com/2012/08/15/building-visualsfm-on-ubuntu-12-04-precise-pangolin-desktop-64-bit/)). Roman Hiestand built a graphical user interface around a series of open-source modules that, when combined in a workflow, enables you to experiment with photogrammetry. With a bit of hacking, we can also make it work with photographs taken from smartphone or tablet.
+
+Download and install the relevant version of [Regard3d](http://www.regard3d.org/) for your operating system.
+
+1. Try the Heistand's [tutorial](http://www.regard3d.org/index.php/documentation/tutorial) using the images of the [Sceaux Castle](http://sourceforge.net/projects/regard3d/files/Demo/OpenMVG/SceauxCastle.zip/download). This tutorial gives you a sense of the basic workflow for using Regard3d.
+
+2. Take your own photographs of an object. Try to capture it from every angle, making sure that there is a high amount of overlap from one photograph to another. Around 20 photographs can be enough to make a model, although more data is normally better. Copy these photos to your computer. A note on smartphone cameras: While many people now have powerful cameras in their pockets in the form of smartphones, these cameras are not in Regard3d's database of cameras and sensor widths. If you're using a smartphone camera, you will have to add this data to the metadata of the images, and then add the 'camera' to the database of cameras. **If you've taken pictures with an actual digital camera, chances are that this information is already present in the Regard3d database**. You'll know if you need to add information if you add a picture set to Regard3d and it says 'NA' beside the picture.
+
+Open Regard3d and start a new project. Add a photoset by selecting the directory where you saved the photos.
+
+Click ok to use the images. **If Regard3d doesn't recognize your camera, check the [Adding metadata to images](Adding metadata to images) section below**.
+
+Click on compute matches. Slide the keypoint density sliders (two sliders) all the way to 'ultra'. You can try with just the default values at first, which is faster, but using 'ultra' means we get as many data points as possible, which can be necessary given our source images. This might take some time. When it is finished, proceed through the next steps as Regard3d presents them to you (the options in the bottom left panel of the program are context-specific. If you want to revisit a previous step and try different settings, select the results from that step in the inspector panel top left to redo).
+
+The final procedure in model generation is to compute the surfaces. When you click on the 'surface' button (having just completed the 'densification' step), make sure to tick off the 'texture' radio button. When this step is complete, you can hit the 'export' button. The model will be in your project folder - .obj, .stl., and .png. To share the model on something like [Sketchfab.com](http://sketchfab.com) zip these three files into a single zip folder. On Sketchfab (or similar services), you would upload the zip folder. These services would then unzip the folder, and their 3d viewers know how to read and display your data.
+
+3. Cleaning up a model with Meshlab
+Building a 3d model takes skill, patience, and practice. No model ever appears 'perfect' on the first try. We can 'fix' a number of issues in a 3d model by opening it in a 3d editing programme. There are many such programmes out there, with various degrees of user-friendliness. One open-source package that is often used is [Meshlab](http://www.meshlab.net/). It is very powerful, but not that intuitive or friendly. It does not 'undo'.
-a) the images themselves
-b) the database of cameras that Regard3d knows.
+Once you have downloaded and installed Meshlab, double-click on the .obj file in your project folder. Meshlab will open and display your model. The exact tools you might wish to use to enhance or clean up your model depends very much on how your model turned out. At the very least, you'll use the 'vertice select' tool (which allows you to draw a box over the offending part) and the 'vertice delete' tool.
-**if you've taken pictures with an actual digital camera, chances are that this information is already present and you're good to go** You'll know if you need to add information if you add a picture set to Regard3d and it says 'NA' beside the picture.
+SG: screenshots, paths to frequently used useful tools in this regard.
#### Adding metadata to images
1. Go to [https://www.sno.phy.queensu.ca/~phil/exiftool/index.html](https://www.sno.phy.queensu.ca/~phil/exiftool/index.html) and download the version of the Exiftool appropriate to your computer.
- **Windows users** you need to fully extract the tool from the zipped download. **THEN** you need to rename the file to just `exiftool.exe`. When you extract it, the tool name is `exiftool(-k).exe`. Delete the `(-k)` in the file name.
- - You need to add data about the camera make, camera model, and focal length to the image files themselves -
- - **Move** the file `exiftool.exe` to the folder where your images are.
+ - **Move** the file `exiftool.exe` to the folder where your images are.
- **Mac users** Unzip if you need to, double click on the dmg file, follow the prompts. You're good to go.
2. Navigate to where your images are store. Windows users, search your machine for `command prompt`. Mac users, search your machine for `terminal`. Run that program. This opens up a window where you can type commands into your computer. You use the `cd` command to 'change directories', followed by the exact location (path) of your images. On a PC it'll probably look something like `cd c:\users\yourname\documents\myimages`. When you're in the location, `dir` will show you everything in that director. Mac users, the command `ls` will list the directory contents. Make sure you're in the right location, (and windows users, that `exiftool.exe` is in that directory).
3. The following commands will add the required metadata to your images. Note that each command is saying, in effect, exiftool, change the following metadata field to this new setting for the following image. the `*.jpeg` means, every single jpeg in this folder. **NB** if your files end with .jpg, you'd use `.jpg`, right?
- - ```exiftool -FocalLength="3.97" *.jpeg``` . <- sets the focal length of your image at 3.97 mm. You should search for your cellphone make online to see if you can find the actual measurement. If you can't find it, 3.97 is probably close enough.
- - ```exiftool -Make="CameraMake" *.jpeg``` <- you can use whatever value you want instead of `CameraMake`. E.g., `myphone` works.
- - ```exiftool -Model="CameraModel" *.jpeg``` <- you can use whatever value you want.
- - If all goes according to plan, the computer will report the number of images modified. Exiftool also makes a copy of your images with the new file extension, `.jpeg_original` so that if something goes wrong, you can delete the new files and restore the old ones by changing their file names (eg, remove `_original` from the name).
-4. Regard3d looks for that metadata in order to do the calculations that generate the point cloud from which the model is created. It needs the focal length, and it needs the size of the image sensor to work. It reads the metadata on make and model and compares it against a database of cameras to get the size of the image sensor plate. Oddly enough, this information is **not** encoded in the image metadata, which is why we need the database. This database is just a text file that uses commas to delimit the fields of information. The pattern looks like this: ```make;model;width-in-mm```. EG: `Canon;Canon-sure-shot;6`. So, we find that file, and we add that information at the end of it.
- - **windows users** This information will be at this location: ```C:\Users\[User name]\AppData\Local\Regard3D``` eg, on my PC: ```C:\Users\ShawnGraham\AppData\Local\Regard3D```, and is in the file "sensor_database.csv".
- - ***mac users** Open your finder, and hit shift+command+g and go to `/Applications/Regard3D.app/Contents/Resources`
- - Do not open `sensor_database.csv` with Excel; Excel will add hidden characters and screw everything up. Instead, you need a proper text editor to work with the file (notepad or wordpad are not useful here). One good option is [Sublime Text](https://www.sublimetext.com/). Download, install, and use it to open `sensor_database.csv`
- - Add whatever you put in for camera make and camera model (back in step 3) *exactly* - uppercase/lowercase matters. You can search to find the size of the image sensor for your cell phone camera. Use that info if you can find it; otherwise 6 mm is probably pretty close.
-- ```ShawnsPhone;LG3;6```
-- Save.
+```exiftool -FocalLength="3.97" *.jpeg```
-Now you can open Regard3d, 'add a picture set', select these images, and Regard3d will have all of the necessary metadata with which to work.
+This sets the focal length of your image at 3.97 mm. You should search for your cellphone make online to see if you can find the actual measurement. If you can't find it, 3.97 is probably close enough.
+```exiftool -Make="CameraMake" *.jpeg```
-Build the model
+You can use whatever value you want instead of `CameraMake`. E.g., `myphone` works.
-Open Regard3d and start a new project. Add a photoset by selecting your frames directory. Note that when you
-used the exiftool, the original images were copied within the folder with a new name. Don't select those original
-images. As the images load up, you will see whether or not your metadata is being correctly read. If you get NaN
-under make, model, focal length, or sensor width, revisit step three again carefully. Click ok to use the images.
-Click on compute matches. Slide the keypoint density sliders (two sliders) all the way to 'ultra'. You can try with
-just the default values at first, which is faster, but using 'ultra' means we get as many data points as possible,
-which can be necessary given our source images.
-This might take some time. When it is finished, proceed through the next steps as Regard3d presents them to
-you (the options in the bottom left panel of the program are context-specific. If you want to revisit a previous
-step and try different settings, select the results from that step in the inspector panel top left to redo).
-The final procedure in model generation is to compute the surfaces. When you click on the 'surface' button
-(having just completed the 'densification' step), make sure to tick off the 'texture' radio button. When this step is
-complete, you can hit the 'export' button. The model will be in your project folder - .obj, .stl., and .png. To share
-the model on something like Sketchfab.com zip these three files into a single zip folder. On sketchfab, you upload
-the zip folder.
+ ```exiftool -Model="CameraModel" *.jpeg```
+
+You can use whatever value you want, eg `LG3`.
+If all goes according to plan, the computer will report the number of images modified. Exiftool also makes a copy of your images with the new file extension, `.jpeg_original` so that if something goes wrong, you can delete the new files and restore the old ones by changing their file names (eg, remove `_original` from the name).
+
+4. Regard3d looks for that metadata in order to do the calculations that generate the point cloud from which the model is created. It needs the focal length, and it needs the size of the image sensor to work. It reads the metadata on make and model and compares it against a database of cameras to get the size of the image sensor plate. Oddly enough, this information is **not** encoded in the image metadata, which is why we need the database. This database is just a text file that uses commas to delimit the fields of information. The pattern looks like this: ```make;model;width-in-mm```. EG: `Canon;Canon-sure-shot;6`. So, we find that file, and we add that information at the end of it.
+ - **windows users** This information will be at this location:
+
+ ```C:\Users\[User name]\AppData\Local\Regard3D```
+
+ eg, on my PC:
+
+ ```C:\Users\ShawnGraham\AppData\Local\Regard3D```
+
+and is in the file "sensor_database.csv".
+
+ - ***mac users** Open your finder, and hit shift+command+g and go to
+
+```/Applications/Regard3D.app/Contents/Resources```
+
+ - Do not open `sensor_database.csv` with Excel; Excel will add hidden characters which cause trouble. Instead, you need a proper text editor to work with the file (notepad or wordpad are not useful here). One good option is [Sublime Text](https://www.sublimetext.com/). Download, install, and use it to open `sensor_database.csv`
+ - Add whatever you put in for camera make and camera model (back in step 3) *exactly* - uppercase/lowercase matters. You can search to find the size of the image sensor for your cell phone camera. Use that info if you can find it; otherwise 6 mm is probably pretty close. The line you add to the database then will look something like this:
+
+```myphone;LG3;6```
+
+- Save.
+
+Now you can open Regard3d, 'add a picture set', select these images, and Regard3d will have all of the necessary metadata with which to work.
-Step Five Clean Up
-Double click on the .obj file in your project folder. Meshlab will open and display your model. The exact tools you
-might wish to use to enhance or clean up your model depends very much on how your model turned out. At the
-very least, you'll use the 'vertice select' tool (which allows you to draw a box over the offending part) and the
-'vertice delete' tool. Search the web for help and examples for the effective use of Meshlab.
-link to humcommons piece for the archival build
\ No newline at end of file
diff --git a/04.3-AI-for-archaeology.rmd b/04.3-AI-for-archaeology.rmd
index fddf02c..3aa3dfb 100644
--- a/04.3-AI-for-archaeology.rmd
+++ b/04.3-AI-for-archaeology.rmd
@@ -1,10 +1,10 @@
-## Artificial Intelligence in Digital Archaeology
+## Artificial Intelligence in Digital Archaeology
To speak of 'artificial intelligence' in archaeology may be to speak too soon yet. We do have machine learning in the service of archaeology (neural networks for classificatory purposes, for instance), and there is a well-established body of work in terms of simulation that could fall under the rubric of 'artificial intelligence'.
-Then why use the term? We think it is still useful to use the term because it reminds us that, in using computational power for simulation or image classification we are off-loading some of our expertise and abilities to a non-human actor. In the case of machine learning and neural networks, we really *can't* see inside the 'black box'. But we can examine the training data, for it is in the selection of training data that we introduce biases or agendas into the computation. By thinking of the machine in this case as something non-human, our hope is that we remind you to not accept the results or methods of AI in archaeology blindly, as if the machine was not capable of racist or colonialist results.
+Then why use the term? We think it is still useful to use the term because it reminds us that, in using computational power for simulation or image classification we are off-loading some of our expertise and abilities to a non-human actor. In the case of machine learning and neural networks, we really *can't* see inside the 'black box'. But we can examine the training data, for it is in the selection of training data that we introduce biases or agendas into the computation. By thinking of the machine in this case as something non-human, our hope is that we remind you to not accept the results or methods of AI in archaeology blindly, as if the machine was not capable of racist or colonialist results.
Machine learning: a series of techniques that endeavour to train a computer program to identify and classify data according to some previously determined values. We will in this chapter discuss image recognition using the neural network model trained by Google, the Inception3 model, which we will query using the Tensorflow package.
@@ -12,7 +12,7 @@ Agent based simulation: a series of techniques that create a population of softw
The value of machine learning: it makes us think carefully about what we are looking for and at in the first place; and then it can be scaled massively.
-The value of agent-based modeling: it forces us to think carefully about what it is we think actually *happened* in the past such that it can be expressed as a series of contextual rules of behavior, functioning under certain conditions.
+The value of agent-based modeling: it forces us to think carefully about what it is we think actually *happened* in the past such that it can be expressed as a series of contextual rules of behavior, functioning under certain conditions.
### Agent-based modeling (ABM)
@@ -25,7 +25,7 @@ Next, we cover the *why* and *how* of the application of ABM in archaeology, his
Last, we walkthrough the steps involved creating an ABM model. Inspired in the theme of emergence and collapse of territorial integration, we designed a model, the _**Pond Trade model**_, showcasing many elements that are common in ABM models in archaeology. The coupled process of design and programming was intentionally broken down into smaller, progressive steps to illustrate an organized pace of code development, from simple to increasingly complex algorithms. The Pond Trade model is implemented in _**NetLogo**_ (https://ccl.northwestern.edu/netlogo/), which is free, easy to install and use, and widely used in academia. The exercise assumes no previous programming knowledge, though practice with NetLogo is required to fully undertand the later versions of the model.
-Throughout this section, several concepts will probably sound *alien* for most history and archaeology students; and some may remain obscure long after completing this lesson. Beyond any practical application, ABM has strong roots in mathematics and logic. _**Don't panic!**_ Most ABM modelers don't go through formal introductions nor are well-versed in algebra and calculus. As in most digital approaches in humanities and social sciences, ABM is mostly done by self-taught experts with hybrid and tortuous academic profiles. Also, because ABM modellers in archaeology are not often computer scientists, the community lacks conventions about how to communicate models and simulation results, though proposals do exist (**REF ODD**). In this sense, the content of this section should NOT be considered the standard among the ABM-archaeology community.
+Throughout this section, several concepts will probably sound *alien* for most history and archaeology students; and some may remain obscure long after completing this lesson. Beyond any practical application, ABM has strong roots in mathematics and logic. _**Don't panic!**_ Most ABM modelers don't go through formal introductions nor are well-versed in algebra and calculus. As in most digital approaches in humanities and social sciences, ABM is mostly done by self-taught experts with hybrid and tortuous academic profiles. Also, because ABM modellers in archaeology are not often computer scientists, the community lacks conventions about how to communicate models and simulation results, though proposals do exist (**REF ODD**). In this sense, the content of this section should NOT be considered the standard among the ABM-archaeology community.
To the date, there is no easy way to begin doing ABM. We encourage students to engage in modelling the sooner, the better, by following our exercises, other tutorials, or their interests and creative anxieties. The full comprehension of ABM will require years of practice and transdisciplinary readings; certainly, a greater mastery in programming. Despite the rarity of ABM in history and archaeology mainstream curricula, there are many publications that offer introductions to ABM in archaeology, writen by authors with different backgrounds and perspectives [e.g., @Breitenecker2015, @Cegielski2016, @Romanowska2015; also visit https://github.com/ArchoLen/The-ABM-in-Archaeology-Bibliography, for an extensive and constantly updated list of references].
@@ -51,7 +51,7 @@ Fine. Models are not only inevitable for any living mind, but they are also the
Formal models--or rather *formalizing* informal models--reduce ambiguity, setting aside part of the risk of being misunderstood. In a formal model everything must be defined, even when the elements of our model are still abstract and general. Our imaginary book about ancient Armenians could be complemented with one or several formal models that crystalize the main features of our informal model(s). For instance, if our informal model about property inheritance includes the nuclear patriarchal family as the main kinship structure, we must define it in *dummy* terms, explaining--to ourselves and others--what "family", "nuclear", and "patriarchal" means, at least in the context of our model. *Do a model assume that family implies co-habitation?* *Do our model consider adopted members?* *What happens if a spouse dies?* As it is often the case, we may end up changing our informal model through the effort of formalization; by realizing, for example, that we can replace "patriarchal" with "patrilineal" because our model do not assume a particular power structure within families.
-Formalization often demand the use of formal logic and some extend of quantification. Seen from the perspective of the human natural language, mathematics and logic are languages for idiots, with a lot of repetitions, fixed vocabulary, and no nuances. Dull. Computational systems function strictly with this kind of language, and they are both empowered and limited by it. A piece of software can potentially do so many increadible things, most of them impossible for the brightest human minds. However, software will be fit for a task as long as we give it enough instructions. Imagine a *formal entity* bumping into a closed door just because nobody explained it how to open doors. So much for Artificial *Inteligence*!
+Formalization often demand the use of formal logic and some extend of quantification. Seen from the perspective of the human natural language, mathematics and logic are languages for idiots, with a lot of repetitions, fixed vocabulary, and no nuances. Dull. Computational systems function strictly with this kind of language, and they are both empowered and limited by it. A piece of software can potentially do so many increadible things, most of them impossible for the brightest human minds. However, software will be fit for a task as long as we give it enough instructions. Imagine a *formal entity* bumping into a closed door just because nobody explained it how to open doors. So much for Artificial *Inteligence*!
A formal definition is a disclaimer saying: *"This is what X means in this model, no more, no less. You can now criticize it at will."* Whitout the formal definition, we would probably spend a chapter reviewing everything we read and thought about "nuclear patriarchal family" and still would not be "at the same page" with all our readers. The terms of this definition are assumptions of the model. At any point, new evidence or insight can suggest us that an assumption is unjustified or unnecessary, and we may change it or remove it from the model.
@@ -77,7 +77,7 @@ ABMs fall somewhere in-between these two spectra, though normally leaning toward
- Define hypotheses, questions or simply the phenomenon of interest (identify the system)
- Define the elements and processes (you believe are) required to address the system (model the system). mention paradox Occam's razor vs. emergence
- Express the chosen elements and processes as computer code (implement the model)
-- Modeling is a process of constant iteration. Each stage is an iteration loop that is often repeated several times before jumping into the next step. The end result of development and exploration of a model may be relatively stable, often in the form of publications. However, this product potentially feeds a new modeling effort (and that is actually the main usefulness of models in the long run).
+- Modeling is a process of constant iteration. Each stage is an iteration loop that is often repeated several times before jumping into the next step. The end result of development and exploration of a model may be relatively stable, often in the form of publications. However, this product potentially feeds a new modeling effort (and that is actually the main usefulness of models in the long run).
- ABM in python, Java, C#, or C++? in R? in NetLogo? (Any language can do it, really). Pros and cons, talk in terms of higher/lower level programming languages (still, they are all quite high-level).
- remember to talk about https://www.openabm.org/
@@ -88,7 +88,7 @@ ABMs fall somewhere in-between these two spectra, though normally leaning toward
- Virtual laboratory for social sciences
- Examples
- Mention the cycles of enthusiasm and skepticism, and the tendency to develop closed circles
-- Challenges and pitfalls
+- Challenges and pitfalls
#### ABM tutorial: the Pond Trade model
@@ -108,7 +108,7 @@ grViz("diagrams/boxes.dot")
```
- Elements: "pond" or water body, settlements, ships, routes, goods.
-- Rules:
+- Rules:
- coastal settlements of variable size around a pond.
- each settlement has a "cultural vector".
- trade ships travel between the settlements.
@@ -141,14 +141,28 @@ SG: code chunks didn't render correctly through the bookdown generation process,
AA: didn't find anything useful about that... Anyhow, I can continue by loading images and leaving r chunks as generic code
-##### Next steps
+## Computer Vision and Archaeology
-### Machine learning for image captioning and other classificatory tasks
+It has become practical in recent years to use neural networks to identify objects, people, and places, in photographs. This use of neural networks _applied to imagery_ in particular has seen rapid advancement since 2012 and the first appearance of 'convolutional' neural networks (@deshpande2016overview provides an accessible guide to this literature). But neural networks in general have appeared sporadically in the archaeological literature since the 1990s; @baxter2014overview provides a useful overview. Recent interesting uses include @benhabiles2016 which uses the approach to enhance pottery databases, and @wang_2017 on stylistic analysis of statuary as an aid to restoration. In this section, we provide a gentle introduction to how convolutional neural networks work as preparation, and then two jupyter binders that may be repurposed or expanded with more data to create actual working classifiers.
-blah
+### Convolutional Neural Networks
-### discussion
+Neural networks are a biological metaphor for a kind of sequence of computations drawing on the architecture of the eye, the optic nerve, and the brain. When the retina of the eye is exposed to light, different structures within the retina react to different aspects of the image being projected against the retina. These 'fire' with greater or lesser strength, depending on what is being viewed. Subsequent neurons will fire if the signal(s) they receive are strong enough. These differential cascades of firing neurons 'light up' the brain in particular, repeatable ways when exposed to different images. Computational neural networks aim to achieve a similar effect. In the same way that a child eventually learns to recognize _this_ pattern of shapes and colour as an 'apple' and _that_ pattern as an 'orange', we can train the computer to 'know' that a particular pattern of activations _should be_ labelled 'apple'.
-blah
+A 'convolutional' neural network begins by 'looking' at an image in terms of its most basic features - curves or areas of contiguous colour. As the information percolates through the network the layers are sensitive to more and more abstraction in the image, some 2048 different dimensions of information. English does not have words to understand _what_ precisely, some (most) of these dimensions are responding to, although if you've seen any of the 'Deep Dream' artworks [SG insert figure here] you are seeing a visualization of some of those dimensions of data. The final layer of neurons predicts from the 2048 dimensions what the image is supposed to be. When we are training such a network, we know at the beginning what the image is of; if at the end, the network does not correctly predict 'apple', this error causes the network to shift its weighting of connections between neurons back through the network ('backpropogation') to increase the chances of a correct response. This process of calculation, guess, evaluation, adjustment goes on until no more improvement seems to occur.
-### exercises
+Neural networks like this can have very complicated architectures to increase their speed, or their accuracy, or some other feature of interest to the researcher. In general, such neural networks are composed of four kinds of layers. The first is the **convolutional** layer. This is a kind of filter that responds to different aspects of an image; it moves across the image from left to right, top to bottom (whence comes the name 'convolutional'). The next layer is the layer that reacts to the information provided by the filter; it is the **activation** layer. The neural network is dealing with an astounding amount of information at this point, and so the third layer, the **pooling** layer does a kind of mathematical reduction or compression to strip out the noise and leave only the most important features in the data. Any particular neural network might have several such 'sandwiches' of neurons arranged in particular ways. The last layer is the **connected** layer, which is the layer with the information concerning the labels. These neurons run a kind of 'vote' on whether or not the 2048-dimension representation of the image 'belongs' to their particular category. This vote is expressed as a percentage, and is typically what we see as the output of a CNN applied to the problem of image identification.
+
+### Applications
+
+Training a neural network to recognize categories of objects is massively computationally intense. Google's Inception3 model - that is, the final state of the neural network Google trained - took the resources of a massive company to put together and millions of images. However, Google _released_ its model to the public. Now anyone can take that _finished_ pattern of weights and neurons and use them in their own applications. But Google didn't train their model on archaeological materials, so it's reasonable to wonder if such a model has any value to us.
+
+It turns out that it does, because of an interesting by-product of the way the model was trained and created. **Transfer learning** allows us to take the high-dimensional ways-of-seeing that the Inception3 model has learned, and apply them to a tweaked final voting layer. We can give the computer mere thousands of images and tell it to learn _these_ categories: and so we can train an image classifier on different kinds of pottery relatively quickly. Google has also released a version of Inception3 called Mobilnet that is much smaller (only 1001 dimensions or ways-of-seeing) and can be used in conjunction with a smartphone. We can use transfer learning on the smaller model as well and create a smartphone application trained to recognize Roman pottery fabrics, for instance.
+
+The focus on identifying objects in photographs does obscure an interesting aspect of the model - that is, there are interesting and useful things that can be done when we dismiss the labeling. The second-to-last layer of the neural network is the numerical representation of the feature-map of the image. We don't need to know what the image is of in order to make use of that information. We can instead feed these representations of the the images into various kinds of k-means, nearest-neighbour, t-sne, or other kinds of statistical tools to look for pattern and structure in the data. If our images are from tourist photos uploaded to flickr of archaeological sites, we might use such tools to understand how tourists are framing their photos (and so, their archaeological consciousness). Graham and Huffer (2018) are using this tool to identify visual tropes in the photographs connected to the communities of people who buy, sell, and collect photos of, human remains on Instagram. Historians are using this approach to understand patterns in 19th century photographs; others are looking at the evolution of advertising in print media.
+
+### Exercises
+
+1. Build an image classifier. The code for this exercise is [in our repo](https://github.com/o-date/image-classifier); [launch the binder](https://mybinder.org/v2/gh/o-date/image-classifier/master) and work carefully through the steps. Pay attention to the various 'flags' that you can set for the training script. Google them; what do they do? Can you improve the speed of the transfer learning? The accuracy? Use what you've learned in section 2.5 to retrieve more data upon which you might build a classifier (hint: there's a script in the repo that might help you with that).
+
+2. Classify similar images. The code for this exercise is in [Shawn Graham's repo](); [launch the binder](http://mybinder.org/v2/gh/shawngraham/bindr-test-Identifying-Similar-Images-with-TensorFlow/master) and work through the steps. Add more image data so that the results are clearer.
diff --git a/50-open-review.Rmd b/50-open-review.Rmd
deleted file mode 100644
index 696de3d..0000000
--- a/50-open-review.Rmd
+++ /dev/null
@@ -1,13 +0,0 @@
-# Open Review {#open}
-
-We are writing this textbook in the open, warts and all. We would be glad of all feedback, which you can make by using the Hypothes.is annotation bar on the right hand side of the screen. You will need an [Hypothes.is](http://web.hypothes.is) account. Then, when you spot something that needs work or is in error or could be better (or even you just want to leave us an encouraging word) highlight the relevant text, hit the 'annotate' button, and leave your remark. Please also use the tagging function in Hypothesis to tag your comment with 'ODATE' so that we can collect the annotations together.
-
-All annotations are public, and will be visible to other visitors on this site. If you have any questions about this, please get in touch with Shawn: shawn dot graham at carleton dot ca.
-
-If you are comfortable with Git and Github, feel free to fork the source code, or leave issues or pull requests as appropriate.
-
-## Contribute {#contribute}
-
-While the majority of this book is being written by Graham, Gupta, Carter and Compton, we know that we are leaving a great deal of material un-discussed. We would be delighted to consider additions to this present work, if you have particular expertise that you would like to share. As you can see, many sections in this work have yet to be written, and so we would be happy to consider contributions aimed there as well. Keep in mind that we are writing for an introductory audience, and that we are writing for a linux-based environment. Whether you are an academic, a professional archaeologist, a graduate student, or a friend of archaeology more generally, we'd be delighted to hear from you.
-
-Please write to Shawn at shawn dot graham at carleton dot ca to discuss your idea and how it might fit into the overall arc of the present work by **January 31st 2018**. The primary authors will discuss whether or not to invite a full draft. A full draft will need to be submitted by **March 15th 2018**. We will then offer feedback. The piece will go up on this draft site by the end of the month, whereupon it will enjoy the same open review as the other parts. Accepted contributors will be listed as full authors, eg 'Graham, Gupta, Carter, Compton, YOUR NAME, 2018 The Open Digital Archaeology Textbook Environment'.
\ No newline at end of file
diff --git a/book.bib b/book.bib
index 8e235d6..9dc38ef 100644
--- a/book.bib
+++ b/book.bib
@@ -1,7 +1,7 @@
%% This BibTeX bibliography file was created using BibDesk.
%% http://bibdesk.sourceforge.net/
-%% Created for Shawn Graham at 2018-05-21 16:39:26 -0400
+%% Created for Shawn Graham at 2018-07-03 15:12:44 -0400
%% Saved with string encoding Unicode (UTF-8)
@@ -11,6 +11,68 @@ @comment{jabref-meta:
+@article{wang_2017,
+ Author = {Haiyan Wang and Zhongshi He and Yongwen Huang and Dingding Chen and Zexun Zhou},
+ Date-Added = {2018-07-03 18:47:10 +0000},
+ Date-Modified = {2018-07-03 19:12:40 +0000},
+ Doi = {https://doi.org/10.1016/j.culher.2017.03.006},
+ Issn = {1296-2074},
+ Journal = {Journal of Cultural Heritage},
+ Keywords = {Deep convolutional network, Modeling style recognition, Dazu Rock Carvings, VGGNet, Exemplar-based inpainting},
+ Pages = {60 - 71},
+ Title = {Bodhisattva head images modeling style recognition of Dazu Rock Carvings based on deep convolutional network},
+ Url = {http://www.sciencedirect.com/science/article/pii/S1296207417300596},
+ Volume = {27},
+ Year = {2017},
+ Bdsk-Url-1 = {http://www.sciencedirect.com/science/article/pii/S1296207417300596},
+ Bdsk-Url-2 = {https://doi.org/10.1016/j.culher.2017.03.006}}
+
+@article{benhabiles2016,
+ Author = {Benhabiles, Halim and Hedi Tabia},
+ Date-Added = {2018-07-03 18:45:13 +0000},
+ Date-Modified = {2018-07-03 18:46:22 +0000},
+ Journal = {Journal of Electronic Imaging},
+ Title = {Convolutional neural network for pottery retrieval},
+ Url = {https://doi.org/10.1117/1.JEI.26.1.011005},
+ Volume = {26},
+ Year = {2016},
+ Bdsk-Url-1 = {https://doi.org/10.1117/1.JEI.26.1.011005}}
+
+@misc{baxter2014overview,
+ Author = {Baxter, Mike},
+ Date-Added = {2018-07-03 18:39:45 +0000},
+ Date-Modified = {2018-07-03 19:03:56 +0000},
+ Lastchecked = {July 3 2018},
+ Title = {Neural Networks in Archaeology},
+ Url = {https://www.academia.edu/8434624/Neural_networks_in_archaeology},
+ Urldate = {2014},
+ Year = {2014},
+ Bdsk-Url-1 = {https://www.academia.edu/8434624/Neural_networks_in_archaeology}}
+
+@url{deshpande2016overview,
+ Author = {Deshpande, Adit},
+ Date-Added = {2018-07-03 18:38:00 +0000},
+ Date-Modified = {2018-07-03 18:39:08 +0000},
+ Lastchecked = {July 3 2018},
+ Month = {August},
+ Title = {The 9 Deep Learning Papers You Need To Know About},
+ Url = {https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html},
+ Viewport = {width=device-width, initial-scale=1, minimum-scale=1, maximum-scale=1, user-scalable=0},
+ Year = {2016},
+ Bdsk-Url-1 = {https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html}}
+
+@article{huffer2018fleshing,
+ Author = {Huffer, Damien and Graham, Shawn},
+ Date-Added = {2018-07-03 18:21:50 +0000},
+ Date-Modified = {2018-07-03 18:22:50 +0000},
+ Journal = {Journal of Computer Applications in Archaeology},
+ Number = {1},
+ Pages = {55--63},
+ Publisher = {Ubiquity Press, Ltd.},
+ Title = {Fleshing Out the Bones: Studying the Human Remains Trade with Tensorflow and Inception},
+ Volume = {1},
+ Year = {2018}}
+
@article{Costopoulos_2016,
Author = {Costopoulos, Andre},
Date-Added = {2018-05-21 16:25:47 +0000},
diff --git a/book/about-the-authors.html b/book/about-the-authors.html
index 844cf6b..fedc81c 100644
--- a/book/about-the-authors.html
+++ b/book/about-the-authors.html
@@ -25,7 +25,7 @@
-
+
@@ -73,10 +73,12 @@
Shawn Graham trained in Roman archaeology but has become over the years a digital archaeologist and digital humanist. He is currently an associate professor in the Department of History at Carleton University in Ottawa Canada. He keeps an open lab notebook of his research and experiments in digital history and archaeology at his research blog, electricarchaeology. He can be found on Twitter at electricarchaeo. His teaching explores methods and digital approaches at all levels, including seminars in the collaborative MA Digital Humanities program as well as in the MA Public History program. His research interests include augmented reality and virtual reality in the service of landscape archaeology, video games about the past, and agent based modeling.
+
Shawn Graham trained in Roman archaeology but has become over the years a digital archaeologist and digital humanist. He is currently an associate professor in the Department of History at Carleton University in Ottawa Canada. He keeps an open lab notebook of his research and experiments in digital history and archaeology at his research blog, electricarchaeology. He can be found on Twitter at electricarchaeo.
Neha Gupta
@@ -240,6 +233,14 @@
Michael Carter
Beth Compton
Beth Compton is a PhD student at the University of Western Ontario’s Department of Anthropology. Her research examines critically the implications and assumptions of digital approaches to repatriation, and uses archaeological ethnographic approaches to evaluate artifact sharing policies. She study archaeological representation and replication, particularly the use of digital photography, 3D modeling, and 3D printing. She is interested in all aspects of meaning-making and knowledge sharing in digital archaeological heritage practice. She is a Co-Founder of the DH Maker Bus project and is an Ontario Trillium Scholar.
+
+
Jolene Smith
+
Jolene Smith is an archaeologist who manages statewide digital archives and archaeological data at the Virginia Department of Historic Resources. She is interested in open data, preservation, and developing easy-to-use tools to help small organizations create, use, and manage archaeological data.
+
+
+
0.0.1 Andreas Angourakis
+
Andreas Angourakis is a PhD student at the Department of History and Archaeology, University of Barcelona. He is a generalist that underwent training in humanistic disciplines, social sciences, and biology. He is self-taught in software and programming languages, including R and NetLogo. His research in archaeology has been focused on the development of simulation models of socio-ecological dynamics in the past, mainly using agent-based modeling, and assembling multivariate statistical protocols for analyzing and interpreting archaeological data. As digital archaeology’s bones of the trade, he believes that creativity and science are unmistakably intertwined. He can be found on Twitter as (???)(https://twitter.com/AndrosSpica), GitHub as Andros-Spica, and is present on both Research Gate and Academia.
2.2 Arranging and Storing Data for the Long Haul (Databases!)
+
Once you’ve collected data according to your plan and structure, you’ll need a way to connect it all together in a way that allows you and other researchers to use it, hopefully for some time to come.
+
So what is a database and which one should you use?
+
Choosing a format: A lot of people have a lot of opinions on good database platforms, horrible ones, and everything in between. But there is no universally right answer and a lot depends on technical resources and level of experience. That is to say, in many cases, a relational database is the strongest option, but handling information well in spreadsheets may met your needs and the needs of other researchers if implemented well. Our organizations may dictate software platforms we’re permitted to use, and often that’s Microsoft Access or Excel.
+
+
2.2.1 discussion
+
+
+
2.2.2 exercises
+
Data Organization in Spreadsheets for Social Scientists
SG: code chunks didn’t render correctly through the bookdown generation process, which is baffling. check the original bookdown book for how to sort this out. removed the executable code chunk, put it into a different script, ran it, copied the output here. this is not optimal.
AA: didn’t find anything useful about that… Anyhow, I can continue by loading images and leaving r chunks as generic code
-
-
4.3.1.4.4 Next steps
-
-
-
-
4.3.2 Machine learning for image captioning and other classificatory tasks
[todo] + simple api call to chronicling america + more complex to open context + some simple plots in the data from open context + calling leaflet from notebook, mapping open context <- but maybe do this in 3.4
It has become practical in recent years to use neural networks to identify objects, people, and places, in photographs. This use of neural networks applied to imagery in particular has seen rapid advancement since 2012 and the first appearance of ‘convolutional’ neural networks (Deshpande (2016) provides an accessible guide to this literature). But neural networks in general have appeared sporadically in the archaeological literature since the 1990s; Baxter (2014) provides a useful overview. Recent interesting uses include Benhabiles and Tabia (2016) which uses the approach to enhance pottery databases, and Wang et al. (2017) on stylistic analysis of statuary as an aid to restoration. In this section, we provide a gentle introduction to how convolutional neural networks work as preparation, and then two jupyter binders that may be repurposed or expanded with more data to create actual working classifiers.
+
+
4.2.1 Convolutional Neural Networks
+
Neural networks are a biological metaphor for a kind of sequence of computations drawing on the architecture of the eye, the optic nerve, and the brain. When the retina of the eye is exposed to light, different structures within the retina react to different aspects of the image being projected against the retina. These ‘fire’ with greater or lesser strength, depending on what is being viewed. Subsequent neurons will fire if the signal(s) they receive are strong enough. These differential cascades of firing neurons ‘light up’ the brain in particular, repeatable ways when exposed to different images. Computational neural networks aim to achieve a similar effect. In the same way that a child eventually learns to recognize this pattern of shapes and colour as an ‘apple’ and that pattern as an ‘orange’, we can train the computer to ‘know’ that a particular pattern of activations should be labelled ‘apple’.
+
A ‘convolutional’ neural network begins by ‘looking’ at an image in terms of its most basic features - curves or areas of contiguous colour. As the information percolates through the network the layers are sensitive to more and more abstraction in the image, some 2048 different dimensions of information. English does not have words to understand what precisely, some (most) of these dimensions are responding to, although if you’ve seen any of the ‘Deep Dream’ artworks [SG insert figure here] you are seeing a visualization of some of those dimensions of data. The final layer of neurons predicts from the 2048 dimensions what the image is supposed to be. When we are training such a network, we know at the beginning what the image is of; if at the end, the network does not correctly predict ‘apple’, this error causes the network to shift its weighting of connections between neurons back through the network (‘backpropogation’) to increase the chances of a correct response. This process of calculation, guess, evaluation, adjustment goes on until no more improvement seems to occur.
+
Neural networks like this can have very complicated architectures to increase their speed, or their accuracy, or some other feature of interest to the researcher. In general, such neural networks are composed of four kinds of layers. The first is the convolutional layer. This is a kind of filter that responds to different aspects of an image; it moves across the image from left to right, top to bottom (whence comes the name ‘convolutional’). The next layer is the layer that reacts to the information provided by the filter; it is the activation layer. The neural network is dealing with an astounding amount of information at this point, and so the third layer, the pooling layer does a kind of mathematical reduction or compression to strip out the noise and leave only the most important features in the data. Any particular neural network might have several such ‘sandwiches’ of neurons arranged in particular ways. The last layer is the connected layer, which is the layer with the information concerning the labels. These neurons run a kind of ‘vote’ on whether or not the 2048-dimension representation of the image ‘belongs’ to their particular category. This vote is expressed as a percentage, and is typically what we see as the output of a CNN applied to the problem of image identification.
+
+
+
4.2.2 Applications
+
Training a neural network to recognize categories of objects is massively computationally intense. Google’s Inception3 model - that is, the final state of the neural network Google trained - took the resources of a massive company to put together and millions of images. However, Google released its model to the public. Now anyone can take that finished pattern of weights and neurons and use them in their own applications. But Google didn’t train their model on archaeological materials, so it’s reasonable to wonder if such a model has any value to us.
+
It turns out that it does, because of an interesting by-product of the way the model was trained and created. Transfer learning allows us to take the high-dimensional ways-of-seeing that the Inception3 model has learned, and apply them to a tweaked final voting layer. We can give the computer mere thousands of images and tell it to learn these categories: and so we can train an image classifier on different kinds of pottery relatively quickly. Google has also released a version of Inception3 called Mobilnet that is much smaller (only 1001 dimensions or ways-of-seeing) and can be used in conjunction with a smartphone. We can use transfer learning on the smaller model as well and create a smartphone application trained to recognize Roman pottery fabrics, for instance.
+
The focus on identifying objects in photographs does obscure an interesting aspect of the model - that is, there are interesting and useful things that can be done when we dismiss the labeling. The second-to-last layer of the neural network is the numerical representation of the feature-map of the image. We don’t need to know what the image is of in order to make use of that information. We can instead feed these representations of the the images into various kinds of k-means, nearest-neighbour, t-sne, or other kinds of statistical tools to look for pattern and structure in the data. If our images are from tourist photos uploaded to flickr of archaeological sites, we might use such tools to understand how tourists are framing their photos (and so, their archaeological consciousness). Graham and Huffer (2018) are using this tool to identify visual tropes in the photographs connected to the communities of people who buy, sell, and collect photos of, human remains on Instagram. Historians are using this approach to understand patterns in 19th century photographs; others are looking at the evolution of advertising in print media.
+
+
+
4.2.3 Exercises
+
+
Build an image classifier. The code for this exercise is in our repo; launch the binder and work carefully through the steps. Pay attention to the various ‘flags’ that you can set for the training script. Google them; what do they do? Can you improve the speed of the transfer learning? The accuracy? Use what you’ve learned in section 2.5 to retrieve more data upon which you might build a classifier (hint: there’s a script in the repo that might help you with that).
+
Classify similar images. The code for this exercise is in Shawn Graham’s repo; launch the binder and work through the steps. Add more image data so that the results are clearer.
Benhabiles, Halim, and Hedi Tabia. 2016. “Convolutional Neural Network for Pottery Retrieval.” Journal of Electronic Imaging 26. https://doi.org/10.1117/1.JEI.26.1.011005.
+
+
+
Wang, Haiyan, Zhongshi He, Yongwen Huang, Dingding Chen, and Zexun Zhou. 2017. “Bodhisattva Head Images Modeling Style Recognition of Dazu Rock Carvings Based on Deep Convolutional Network.” Journal of Cultural Heritage 27: 60–71. doi:https://doi.org/10.1016/j.culher.2017.03.006.
In recent years, faster and more powerful computers have made it feasible to do complex 3d model building by extracting points of overlap in multiple photographs, then extrapolating from the camera metadata embedded in those images the distance from the points to the camera’s image sensor. This information allows the reconstruction of where those points were in space relative to the camera. Thus astonishingly good 3d models can be created at rather low cost.
Laser scanning, on the other hand, involves shooting rays of light onto an object (or space) and counting the time it takes for the light to return to the scanner. Laser scanners are able therefore to take detailed micro-millimetre scans of an object’s surface and texture. For some archaeological purposes, laser scanning is to be preferred. For other purposes, 3d photogrammetry or ‘structure from motion’ (sfm) is entirely appropriate, and the level of resolution good enough
In this chapter, we’ll cover some of the basic principles of how sfm works while pointing you to more detailed discussions. We also provide links in the ‘further readings’ to some recent case studies using 3d photogrammetry in novel ways.
-
Basic principles
+
+
4.1.1 Basic principles
image capture
image matching
@@ -257,59 +251,220 @@
4.1 3D Photogrammetry & Struc
download to your computer; clean up
-
-
4.1.1 discussion
+
from regard3d: To obtain a 3D model, the following steps are performed:
+
For each image, features (sometimes also called keypoints) are detected. Features are points in an object that have a high probability to be found in different images of the same object, for example corners, edges etc. Regard3D uses A-KAZE for this purpose. For each feature, a mathematical descriptor is calculated. This descriptor has the characteristic that descriptors of the same point in an object in different images (seen from different viewpoints) is similar. Regard3D uses LIOP (Local Intensity Order Pattern) for this purpose. The descriptors from different images are matched and geometrically filtered. The result of this step is a collection of matches between each image pair. Now “tracks” are calculated. For each feature that is part of a match in an image pair, it is searched also in other images. A track is generated from features if these features satisfy some conditions, for example a track is seen in at least 3 images. The next step is the triangulation phase. All the matches of all the image pairs are used to calculate: The 3D position and characteristic of the “camera”, i.e. where each image was shot and the visual characteristics of the camera The 3D position of each “track” is calculated The result of the triangulation phase is a sparse point cloud. In order to obtain a more dense point cloud (“densification”), several algorithms can be used. The last step is called “Surface generation”. The point clouds are used to generate a surface, either with colored vertices or with a texture.
4.1.2 exercises
-
-
-
4.1.3 nuts and bolts
-
In order to use something like Regard3d to build models from pictures you’ve taken with your cellphone, you need to add certain information to
-
-
the images themselves
-
the database of cameras that Regard3d knows.
+
While there are command-line applications (like VSFM) for photogrammetry, running such things from a Jupyter notebook does not work very well. VSFM can be installed in something like DHBox (but it’s not for the faint-of-heart, see eg this). Roman Hiestand built a graphical user interface around a series of open-source modules that, when combined in a workflow, enables you to experiment with photogrammetry. With a bit of hacking, we can also make it work with photographs taken from smartphone or tablet.
+
Download and install the relevant version of Regard3d for your operating system.
+
+
Try the Heistand’s tutorial using the images of the Sceaux Castle. This tutorial gives you a sense of the basic workflow for using Regard3d.
+
Take your own photographs of an object. Try to capture it from every angle, making sure that there is a high amount of overlap from one photograph to another. Around 20 photographs can be enough to make a model, although more data is normally better. Copy these photos to your computer. A note on smartphone cameras: While many people now have powerful cameras in their pockets in the form of smartphones, these cameras are not in Regard3d’s database of cameras and sensor widths. If you’re using a smartphone camera, you will have to add this data to the metadata of the images, and then add the ‘camera’ to the database of cameras. If you’ve taken pictures with an actual digital camera, chances are that this information is already present in the Regard3d database. You’ll know if you need to add information if you add a picture set to Regard3d and it says ‘NA’ beside the picture.
+
+
Open Regard3d and start a new project. Add a photoset by selecting the directory where you saved the photos.
+
Click ok to use the images. If Regard3d doesn’t recognize your camera, check the Adding metadata to images section below.
+
Click on compute matches. Slide the keypoint density sliders (two sliders) all the way to ‘ultra’. You can try with just the default values at first, which is faster, but using ‘ultra’ means we get as many data points as possible, which can be necessary given our source images. This might take some time. When it is finished, proceed through the next steps as Regard3d presents them to you (the options in the bottom left panel of the program are context-specific. If you want to revisit a previous step and try different settings, select the results from that step in the inspector panel top left to redo).
+
The final procedure in model generation is to compute the surfaces. When you click on the ‘surface’ button (having just completed the ‘densification’ step), make sure to tick off the ‘texture’ radio button. When this step is complete, you can hit the ‘export’ button. The model will be in your project folder - .obj, .stl., and .png. To share the model on something like Sketchfab.com zip these three files into a single zip folder. On Sketchfab (or similar services), you would upload the zip folder. These services would then unzip the folder, and their 3d viewers know how to read and display your data.
+
+
Cleaning up a model with Meshlab Building a 3d model takes skill, patience, and practice. No model ever appears ‘perfect’ on the first try. We can ‘fix’ a number of issues in a 3d model by opening it in a 3d editing programme. There are many such programmes out there, with various degrees of user-friendliness. One open-source package that is often used is Meshlab. It is very powerful, but not that intuitive or friendly. It does not ‘undo’.
-
if you’ve taken pictures with an actual digital camera, chances are that this information is already present and you’re good to go You’ll know if you need to add information if you add a picture set to Regard3d and it says ‘NA’ beside the picture.
+
Once you have downloaded and installed Meshlab, double-click on the .obj file in your project folder. Meshlab will open and display your model. The exact tools you might wish to use to enhance or clean up your model depends very much on how your model turned out. At the very least, you’ll use the ‘vertice select’ tool (which allows you to draw a box over the offending part) and the ‘vertice delete’ tool.
+
SG: screenshots, paths to frequently used useful tools in this regard.
Windows users you need to fully extract the tool from the zipped download. THEN you need to rename the file to just exiftool.exe. When you extract it, the tool name is exiftool(-k).exe. Delete the (-k) in the file name.
-
You need to add data about the camera make, camera model, and focal length to the image files themselves -
Move the file exiftool.exe to the folder where your images are.
Mac users Unzip if you need to, double click on the dmg file, follow the prompts. You’re good to go.
Navigate to where your images are store. Windows users, search your machine for command prompt. Mac users, search your machine for terminal. Run that program. This opens up a window where you can type commands into your computer. You use the cd command to ‘change directories’, followed by the exact location (path) of your images. On a PC it’ll probably look something like cd c:\users\yourname\documents\myimages. When you’re in the location, dir will show you everything in that director. Mac users, the command ls will list the directory contents. Make sure you’re in the right location, (and windows users, that exiftool.exe is in that directory).
-
The following commands will add the required metadata to your images. Note that each command is saying, in effect, exiftool, change the following metadata field to this new setting for the following image. the *.jpeg means, every single jpeg in this folder. NB if your files end with .jpg, you’d use .jpg, right?
+
The following commands will add the required metadata to your images. Note that each command is saying, in effect, exiftool, change the following metadata field to this new setting for the following image. the *.jpeg means, every single jpeg in this folder. NB if your files end with .jpg, you’d use .jpg, right?
-
-
exiftool -FocalLength="3.97" *.jpeg . <- sets the focal length of your image at 3.97 mm. You should search for your cellphone make online to see if you can find the actual measurement. If you can’t find it, 3.97 is probably close enough.
-
exiftool -Make="CameraMake" *.jpeg <- you can use whatever value you want instead of CameraMake. E.g., myphone works.
-
exiftool -Model="CameraModel" *.jpeg <- you can use whatever value you want.
-
-
If all goes according to plan, the computer will report the number of images modified. Exiftool also makes a copy of your images with the new file extension, .jpeg_original so that if something goes wrong, you can delete the new files and restore the old ones by changing their file names (eg, remove _original from the name).
-
+
exiftool -FocalLength="3.97" *.jpeg
+
This sets the focal length of your image at 3.97 mm. You should search for your cellphone make online to see if you can find the actual measurement. If you can’t find it, 3.97 is probably close enough.
+
exiftool -Make="CameraMake" *.jpeg
+
You can use whatever value you want instead of CameraMake. E.g., myphone works.
+
exiftool -Model="CameraModel" *.jpeg
+
You can use whatever value you want, eg LG3.
+
If all goes according to plan, the computer will report the number of images modified. Exiftool also makes a copy of your images with the new file extension, .jpeg_original so that if something goes wrong, you can delete the new files and restore the old ones by changing their file names (eg, remove _original from the name).
Regard3d looks for that metadata in order to do the calculations that generate the point cloud from which the model is created. It needs the focal length, and it needs the size of the image sensor to work. It reads the metadata on make and model and compares it against a database of cameras to get the size of the image sensor plate. Oddly enough, this information is not encoded in the image metadata, which is why we need the database. This database is just a text file that uses commas to delimit the fields of information. The pattern looks like this: make;model;width-in-mm. EG: Canon;Canon-sure-shot;6. So, we find that file, and we add that information at the end of it.
-
windows users This information will be at this location: C:\Users\[User name]\AppData\Local\Regard3D eg, on my PC: C:\Users\ShawnGraham\AppData\Local\Regard3D, and is in the file “sensor_database.csv”.
-
*mac users Open your finder, and hit shift+command+g and go to /Applications/Regard3D.app/Contents/Resources
-
Do not open sensor_database.csv with Excel; Excel will add hidden characters and screw everything up. Instead, you need a proper text editor to work with the file (notepad or wordpad are not useful here). One good option is Sublime Text. Download, install, and use it to open sensor_database.csv
-
Add whatever you put in for camera make and camera model (back in step 3) exactly - uppercase/lowercase matters. You can search to find the size of the image sensor for your cell phone camera. Use that info if you can find it; otherwise 6 mm is probably pretty close.
-
ShawnsPhone;LG3;6
-
Save.
+
windows users This information will be at this location:
+
+
C:\Users\[User name]\AppData\Local\Regard3D
+
eg, on my PC:
+
C:\Users\ShawnGraham\AppData\Local\Regard3D
+
and is in the file “sensor_database.csv”.
+
+
*mac users Open your finder, and hit shift+command+g and go to
-
Now you can open Regard3d, ‘add a picture set’, select these images, and Regard3d will have all of the necessary metadata with which to work.
-
Build the model
-
Open Regard3d and start a new project. Add a photoset by selecting your frames directory. Note that when you used the exiftool, the original images were copied within the folder with a new name. Don’t select those original images. As the images load up, you will see whether or not your metadata is being correctly read. If you get NaN under make, model, focal length, or sensor width, revisit step three again carefully. Click ok to use the images. Click on compute matches. Slide the keypoint density sliders (two sliders) all the way to ‘ultra’. You can try with just the default values at first, which is faster, but using ‘ultra’ means we get as many data points as possible, which can be necessary given our source images. This might take some time. When it is finished, proceed through the next steps as Regard3d presents them to you (the options in the bottom left panel of the program are context-specific. If you want to revisit a previous step and try different settings, select the results from that step in the inspector panel top left to redo). The final procedure in model generation is to compute the surfaces. When you click on the ‘surface’ button (having just completed the ‘densification’ step), make sure to tick off the ‘texture’ radio button. When this step is complete, you can hit the ‘export’ button. The model will be in your project folder - .obj, .stl., and .png. To share the model on something like Sketchfab.com zip these three files into a single zip folder. On sketchfab, you upload the zip folder.
-
Step Five Clean Up Double click on the .obj file in your project folder. Meshlab will open and display your model. The exact tools you might wish to use to enhance or clean up your model depends very much on how your model turned out. At the very least, you’ll use the ‘vertice select’ tool (which allows you to draw a box over the offending part) and the ‘vertice delete’ tool. Search the web for help and examples for the effective use of Meshlab.
-
link to humcommons piece for the archival build
+
/Applications/Regard3D.app/Contents/Resources
+
+
Do not open sensor_database.csv with Excel; Excel will add hidden characters which cause trouble. Instead, you need a proper text editor to work with the file (notepad or wordpad are not useful here). One good option is Sublime Text. Download, install, and use it to open sensor_database.csv
+
Add whatever you put in for camera make and camera model (back in step 3) exactly - uppercase/lowercase matters. You can search to find the size of the image sensor for your cell phone camera. Use that info if you can find it; otherwise 6 mm is probably pretty close. The line you add to the database then will look something like this:
+
+
+- Save.
+
+Now you can open Regard3d, 'add a picture set', select these images, and Regard3d will have all of the necessary metadata with which to work.
+
+
+
+
+
+<!--chapter:end:04.1-3d-photogrammetry.rmd-->
+
+## 3D Printing, the Internet of Things and “Maker” Archaeology
+
+yay
+
+### discussion
+
+### exercises
+
+<!--chapter:end:04.2-3d-printing.rmd-->
+
+
+
+## Artificial Intelligence in Digital Archaeology
+
+To speak of 'artificial intelligence' in archaeology may be to speak too soon yet. We do have machine learning in the service of archaeology (neural networks for classificatory purposes, for instance), and there is a well-established body of work in terms of simulation that could fall under the rubric of 'artificial intelligence'.
+
+Then why use the term? We think it is still useful to use the term because it reminds us that, in using computational power for simulation or image classification we are off-loading some of our expertise and abilities to a non-human actor. In the case of machine learning and neural networks, we really *can't* see inside the 'black box'. But we can examine the training data, for it is in the selection of training data that we introduce biases or agendas into the computation. By thinking of the machine in this case as something non-human, our hope is that we remind you to not accept the results or methods of AI in archaeology blindly, as if the machine was not capable of racist or colonialist results.
+
+Machine learning: a series of techniques that endeavour to train a computer program to identify and classify data according to some previously determined values. We will in this chapter discuss image recognition using the neural network model trained by Google, the Inception3 model, which we will query using the Tensorflow package.
+
+Agent based simulation: a series of techniques that create a population of software 'agents' who are programmed with contextual rules (e.g., *if this happens, do that*) governing the behaviour of individual agents. The context can be both in terms of the simulated environment (GIS data, for instance) or the social environment (social relationships as a network).<!---The simulation iterates--> Simulation experiments iterate over multiple combinations of parameters' values<!---an entire landscape of possible variables-->, <!---creating--> the 'behaviour space'. <!---which is--> Simulation results are then used by the investigator to <!---understand the emergent behaviour in the situation, to understand better the possible range of situations that could account for the observed phenomenon in the 'real world'--> explain the 'real world' phenomenon as an emergency of a population of agents following a set of rules under certain situations.
+
+The value of machine learning: it makes us think carefully about what we are looking for and at in the first place; and then it can be scaled massively.
+
+The value of agent-based modeling: it forces us to think carefully about what it is we think actually *happened* in the past such that it can be <!---encoded as a series of individual-level instructions--> expressed as a series of contextual rules of behavior, functioning under certain conditions.
+
+
+### Agent-based modeling (ABM)
+
+This section is an overview of agent-based modeling (**_ABM_**), as it is used within archaeology, anthropology, and behavioral sciences. Before tackling the append "agent-based", we start with a brief introduction to what "modeling" means. To place ABM in a bigger picture, we introduce the student to several dimensions of the variability of simulation models. We describe the necessary and optional parts of ABM models, including their characteristic type of element, i.e. the agent. At this point, the student must be aware that _**simulation models**_ in our context involve computer code (they are *digital*) and the iteration of processes (they are *dynamic*).
+
+Following the definition of ABM, we describe the process of creating a model or *formalizing* an informal model. Although the reality is messier, this process can be divided into *definition* of research question and phenomenon, *design*, *implementation*, and *verification*. *Validation*, which is considered a fifth step by most of the modeling literature, is regarded here as an extra, lengthy process, often separable from the tasks involved in creating and exploring a model. Furthermore, we introduce two additional steps that are not normally acknowledged: *understanding* and *documenting* the model. In explaining all modeling steps, we keep instructions general enough for them to be valid when dealing with any platform or programming language.
+
+Next, we cover the *why* and *how* of the application of ABM in archaeology, history, and social sciences. We recover some key foundational works, including models that predate the development of ABM as it defined today. Specifically regarding archaeological applications, we illustrate the diversity of ABM approaches by presenting several examples of models with varying goals, spatial and temporal scales, and theoretical backgrounds.
+
+Last, we walkthrough the steps involved creating an ABM model. Inspired in the theme of emergence and collapse of territorial integration, we designed a model, the _**Pond Trade model**_, showcasing many elements that are common in ABM models in archaeology. The coupled process of design and programming was intentionally broken down into smaller, progressive steps to illustrate an organized pace of code development, from simple to increasingly complex algorithms. The Pond Trade model is implemented in _**NetLogo**_ (https://ccl.northwestern.edu/netlogo/), which is free, easy to install and use, and widely used in academia. The exercise assumes no previous programming knowledge, though practice with NetLogo is required to fully undertand the later versions of the model.
+
+Throughout this section, several concepts will probably sound *alien* for most history and archaeology students; and some may remain obscure long after completing this lesson. Beyond any practical application, ABM has strong roots in mathematics and logic. _**Don't panic!**_ Most ABM modelers don't go through formal introductions nor are well-versed in algebra and calculus. As in most digital approaches in humanities and social sciences, ABM is mostly done by self-taught experts with hybrid and tortuous academic profiles. Also, because ABM modellers in archaeology are not often computer scientists, the community lacks conventions about how to communicate models and simulation results, though proposals do exist (**REF ODD**). In this sense, the content of this section should NOT be considered the standard among the ABM-archaeology community.
+
+To the date, there is no easy way to begin doing ABM. We encourage students to engage in modelling the sooner, the better, by following our exercises, other tutorials, or their interests and creative anxieties. The full comprehension of ABM will require years of practice and transdisciplinary readings; certainly, a greater mastery in programming. Despite the rarity of ABM in history and archaeology mainstream curricula, there are many publications that offer introductions to ABM in archaeology, writen by authors with different backgrounds and perspectives [e.g., @Breitenecker2015, @Cegielski2016, @Romanowska2015; also visit https://github.com/ArchoLen/The-ABM-in-Archaeology-Bibliography, for an extensive and constantly updated list of references].
+
+#### What is ABM?
+
+>"Essentially, all models are wrong, but some are useful"
+>George Box, 1987
+>*Empirical Model-Building and Response Surfaces*, p. 424
+
+The first thing to consider is that ABM is about models. A model is a representation of a phenomenon as a set of elements and their relationships; the key term being *representation*. It is a generalization/abstraction/simplification of what we consider as *THE* phenomenon. Think about a pineapple. The mental representation of the phenomenon, "the pineapple", is already a model. It consists of a set of traits or visual cues, and their relative positions (e.g., crown of spicky leafs on top, thorns covering the oval body). Empirically, every pineapple has different visual properties, but we still are able to identify any pineapple we encounter by using this model.
+
+
+
+Another important statement about models is that they are--as any mental process--a phenomenon in its own right. That puts us in the realm of epistemology and cognitive sciences. It follows that one's model is *a* model and that there are many models representing a phenomenon as there are minds that recognise this phenomenon. Observe the diversity of pineapple drawings. Adding further complexity to this picture, minds are constantly changing with the integration of new experiences--creating, modifying, merging, and forgetting models--and we often exchange and contrast alternative models through expressing them to ourselves and others. As the last straw of confusion, the ability for communicating models give them the status of *'material objects'*, with a *existence* of their own, reproduced through learning, checked by the clash of new experiences, and in perpetual change through innovation and creativity. For example, a child with little or no experience with the 'real pineapple' is able to learn--and graciously modify--a model of pineapple.
+
+Well, what do pineapple drawings have in common with doing models in archaeology? You may imagine that, if your mental representation of a pineapple is a model, human minds are bursting with models about everything, including *models of models*, *models within models*, and *models of ourselves*. If an archaeologist, there will be many archaeologically-sensitive models buried deep inside such mental apparatus. *"Why did people in this site buried food remains in small pits?" "They wanted to avoid attracting critters." "They did not stand the smell of decomposition." "One person with high status started doing it and them the rest just followed." "They were offering sustainance to the land spirit."* These explanations derive directly from models of *why people behave as they do*. Often tagged as *social* or *behavioral*, this kind of model is as key in archaeology as it is neglected. Given that archaeological practice orbits around materials, archaeologists tend to relegate social models to a second plane, as if the goal is actually to understand the material remains of human behavior rather than the behavior itself. More importantly, many archaeologists are hardly conscient that they are using social models even when emitting a conservative interpretations. Another common practice is to present authoritative models as mantras while unconsciously hidding the models used. Social models are indeed more permeable to *personal*, *subjective* perspectives--in contrast with a model of stratification, for example--because we use them daily to interact with others, apart from archaeological affairs. Thus, they are strongly mould to our life experiences and beliefs, which vary depending on many socio-cultural factors, including those less obvious such as age and gender. Even though there are shared traits between them, these models are as many and diverse as pineapple drawings.
+
+Why bother then doing models in academia? Ultimately, we are searching for knowledge, staments with at least some *value of truth*, independent of any individual, not a ever-growing collection of models, even if they are *authorative* and well-accepted. Remember that models of a phenomena, as diverse they may be, are contrasted with experiences of that phenomena, which are much less variable. As George Box's famous phrase states, models are NOT equivalent reproductions of a phenomenon, but tools for looking and interacting with it. What we could define as *imaginative models*, such as the smiling pineapple, though still able to exist for other purposes, can be clearly distinguished from models with *vocation of truth*, because they fail to match our sensible experience and are incompatible with other successful models. Those that cannot be distinguished, should be considered *valid*, *competing* models; their study, validation, and improvement being the main goal of scientific endeavours.
+
+Unfortunately, models involving human behavior are less straightforward to validate or dismiss than a "smiley pineapple". Still, when having many competing models, progress can be made by ordering them by 'probability of being true', following many criteria. A warning though: almost-certain models might be eventually proven deciduous, and marginal--even discredited--models can end being mainstream. To be alive--read *useful*--models must be communicated, used, re-used, abused, recycled, hacked, and broken. A well-documented model, preserved in many locations and formats, could as easily be considered *hibernating*, awaiting new evidence for validation or a new generation with a more favorable mentality.
+
+Fine. Models are not only inevitable for any living mind, but they are also the keystone for the generation of knowledge. However, if models can be so simple, spontaneous, and intuitive as a pineapple drawing, what is the need of doing strange models with complicated equations, painstaking specifications, and fathomless designs? Archaeologist use models to define their research agenda and guide their interpretations. A lot of letter has been spent in presenting, discussing, and revisiting archaeological and archaeologicaly-relevant models. Nonetheless, few of these works are able to define unambiguosly the model(s) in question, not because they lack in effort but due to the intrinsic limitation of the media objectifying the model: human, natural, verbal language. For example, we may write a book on how we believe, given known evidence, that kinship relates to property rights among, say, the Armenians contemporaneous to the Early Roman Empire. Our reasoning and description of possible casuistics can be lengthy and impeccable; however, we are bound to be misunderstood at some point, because readers may not share the same concepts and certainly not all connotations that we may have wanted to convey. A significant part of the model in our minds would be writen between the lines or not at all. Depending on their background, readers can potentially miss altogether that we are referring to *a model*, and consider the elements of our model as loose speculations or, worse, as given facts.
+
+Formal models--or rather *formalizing* informal models--reduce ambiguity, setting aside part of the risk of being misunderstood. In a formal model everything must be defined, even when the elements of our model are still abstract and general. Our imaginary book about ancient Armenians could be complemented with one or several formal models that crystalize the main features of our informal model(s). For instance, if our informal model about property inheritance includes the nuclear patriarchal family as the main kinship structure, we must define it in *dummy* terms, explaining--to ourselves and others--what "family", "nuclear", and "patriarchal" means, at least in the context of our model. *Do a model assume that family implies co-habitation?* *Do our model consider adopted members?* *What happens if a spouse dies?* As it is often the case, we may end up changing our informal model through the effort of formalization; by realizing, for example, that we can replace "patriarchal" with "patrilineal" because our model do not assume a particular power structure within families.
+
+Formalization often demand the use of formal logic and some extend of quantification. Seen from the perspective of the human natural language, mathematics and logic are languages for idiots, with a lot of repetitions, fixed vocabulary, and no nuances. Dull. Computational systems function strictly with this kind of language, and they are both empowered and limited by it. A piece of software can potentially do so many increadible things, most of them impossible for the brightest human minds. However, software will be fit for a task as long as we give it enough instructions. Imagine a *formal entity* bumping into a closed door just because nobody explained it how to open doors. So much for Artificial *Inteligence*!
+
+A formal definition is a disclaimer saying: *"This is what X means in this model, no more, no less. You can now criticize it at will."* Whitout the formal definition, we would probably spend a chapter reviewing everything we read and thought about "nuclear patriarchal family" and still would not be "at the same page" with all our readers. The terms of this definition are assumptions of the model. At any point, new evidence or insight can suggest us that an assumption is unjustified or unnecessary, and we may change it or remove it from the model.
+
+
+
+- Why model with ABM? open with the idea that we are always modeling
+- Mention review article 'Why model?'
+
+- def system, model, formal model
+- The origins of computer simulation
+- 3d models and statistical models (e.g., regression) are not simulation models. Simulation implies the passing of time as the driving variable of a model. Other kinds of models may share the goal of inferring new data from a given knowledge or dataset. For instance, we can design a curve that fits the number of archaeological sites in a region for all known periods, and use it to estimate this value during periods not represented in any sites. However, this equation is not representing the mechanisms that are supposely causing the phenomenon, i.e. the distribution of human activity in space and time. The idea of step, iteration, or repetition as the representation of passing time is the keystone of any simulation model.
+- Equation-based modeling, System Dynamics - ODE systems as models (mention Lorenz attractor, end with the idea of iteration of a process)
+- Algorithm-based modeling, Algorithms as behavioral models (mention cellular automata, Van Neumann, Schelling, etc)
+- The variety of algorithm-based models, much coincides with the variety of programming paradigm.
+ a. Flow: spectrum between centralized (iteration of a single process, e.g. event) and distributed (multiple processes are iterated contextually by multiple entities).
+ b. Schedule: spectrum between fully-scheduled (processes are initiated in a predetermined order) and event-driven (processes are initiated given a certain input).
+ABMs fall somewhere in-between these two spectra, though normally leaning towards distributed (e.g., using Monte Carlo methods) and fully-scheduled. Applications in several disciplines (see Wikipedia)
+- Parts of an ABM model
+ (re-visit presentations)
+
+#### How to model with ABM?
+
+- Define hypotheses, questions or simply the phenomenon of interest (identify the system)
+- Define the elements and processes (you believe are) required to address the system (model the system). mention paradox Occam's razor vs. emergence
+- Express the chosen elements and processes as computer code (implement the model)
+- Modeling is a process of constant iteration. Each stage is an iteration loop that is often repeated several times before jumping into the next step. The end result of development and exploration of a model may be relatively stable, often in the form of publications. However, this product potentially feeds a new modeling effort (and that is actually the main usefulness of models in the long run).
+- ABM in python, Java, C#, or C++? in R? in NetLogo? (Any language can do it, really). Pros and cons, talk in terms of higher/lower level programming languages (still, they are all quite high-level).
+- remember to talk about https://www.openabm.org/
+
+#### ABM in Archaeology and Social Sciences
+
+- Emergence... Life game, deterministic chaos
+- Saving the gap between atomism and holism, or individual and social structure
+- Virtual laboratory for social sciences
+- Examples
+- Mention the cycles of enthusiasm and skepticism, and the tendency to develop closed circles
+- Challenges and pitfalls
+
+#### ABM tutorial: the Pond Trade model
+
+brief overview
+disclaimer: MY modelling strategy is highly focused in theory-building (general, explorative), not hypothesis-testing and prediction (specific, data-driven).
+
+##### Phenomena of interest and conceptual model
+
+- phenomena: cycles of growth and collapse, i.e. flutuations in the scale of site occupation, out of phase with environmental change. Focus on coastal settlements around a water body (e.g., lake, bay, sea).
+- Belief or main assumption: topography, transport technology, trade, settlement size and wealth, and cultural exchange are intertwined.
+
+
+
+
library(DiagrammeR) grViz(“diagrams/boxes.dot”)
+
+- Elements: "pond" or water body, settlements, ships, routes, goods.
+- Rules:
+ - coastal settlements of variable size around a pond.
+ - each settlement has a "cultural vector".
+ - trade ships travel between the settlements.
+ - once in their base settlement, ships evaluate all possible trips and choose the one with the greater cost-benefit ratio.
+ - ships carry economic value and cultural traits between the base and destination settlements.
+ - settlements size depends on the economic value they receive from trade.
+ - the economic value produced in a settlement depends on its size.
+ - the number of ships per settlement depends on its size.
+
+##### Implementation
+
+- Implement the model based on general definition (agents, variables, parameters)
+- Walk-through the implementation in NetLogo
+
+Pond Trade model repository: https://github.com/Andros-Spica/PondTrade
+
+
+
+##### Simulation experiments and analysis
+- Simulations and experiment design
+- Analysis and display of results (R example)
+
+
plot(1:10, 1:10) ```
+
SG: code chunks didn’t render correctly through the bookdown generation process, which is baffling. check the original bookdown book for how to sort this out. removed the executable code chunk, put it into a different script, ran it, copied the output here. this is not optimal.
+
AA: didn’t find anything useful about that… Anyhow, I can continue by loading images and leaving r chunks as generic code
Graham ignored security warnings from the main platform’s maintainers
Backups and versioning: there were none.
-
Graham’s fails here are of both type 1 and type 2. In terms of type 2, his failure to keep careful notes on how the various pieces of the project were made to fit together meant that he lacked the framework to understand how he had made the project vulnerable to attack. The actual fail point of the attack - that’s a type 1 fail, but could have been avoided if Graham had participated more in the spirit of open software development, eg, read the security warnings in the developer forum! When Graham realized what had happened to his project, he was faced with two options. One option, having already published a piece on the project that hailed its successes and broad lessons for crowdsourcing cultural heritage, would have been to quietly walked away from the project (perhaps putting up a new website avverring that version 2.0 was coming, pending funding). The other option was to warn folks to beef up the security and backups for their own projects. At the time, crowdsourcing was very much an academic fashion and Graham opted for the second option in that spirit. In doing this, the HeritageCrowd project became a fail of type 3, an artifact for study and reflection. The act of blogging his post-mortem makes this project also an instance of type 5, or the communication of the fail. It is worth pointing out here that the public sharing of this failure is not without issues. As we indicated in the Open Notebook Research & Scholarly Communication section, the venue for sharing what hasn’t worked and the lessons learned is highly contingent on many factors. Graham, as a white male tenure-track academic on the web in 2012, could share openly that things had not worked. As the web has developed in the intervening years, and with the increasing polarization of web discourse into broad ideological camps, it may well not be safe for someone in a more precarious position to share so openly. One must keep in mind one’s own situation and adapt what we argue for here accordingly. Sharing fails can be done with close colleagues, students, specialist forums and so on.
+
Graham’s fails here are of both type 1 and type 2. In terms of type 2, his failure to keep careful notes on how the various pieces of the project were made to fit together meant that he lacked the framework to understand how he had made the project vulnerable to attack. The actual fail point of the attack - that’s a type 1 fail, but could have been avoided if Graham had participated more in the spirit of open software development, eg, read the security warnings in the developer forum! When Graham realized what had happened to his project, he was faced with two options. One option, having already published a piece on the project that hailed its successes and broad lessons for crowdsourcing cultural heritage, would have been to quietly walked away from the project (perhaps putting up a new website averring that version 2.0 was coming, pending funding). The other option was to warn folks to beef up the security and backups for their own projects. At the time, crowdsourcing was very much an academic fashion and Graham opted for the second option in that spirit. In doing this, the HeritageCrowd project became a fail of type 3, an artifact for study and reflection. The act of blogging his post-mortem makes this project also an instance of type 5, or the communication of the fail. It is worth pointing out here that the public sharing of this failure is not without issues. As we indicated in the Open Notebook Research & Scholarly Communication section, the venue for sharing what hasn’t worked and the lessons learned is highly contingent on many factors. Graham, as a white male tenure-track academic on the web in 2012, could share openly that things had not worked. As the web has developed in the intervening years, and with the increasing polarization of web discourse into broad ideological camps, it may well not be safe for someone in a more precarious position to share so openly. One must keep in mind one’s own situation and adapt what we argue for here accordingly. Sharing fails can be done with close colleagues, students, specialist forums and so on.
If we are successful with ODATE, and the ideas of productive fail begin to permeate more widely in the teaching of digital archaeology, then a pedagogy that values fail will with time normalize such ‘negative results’. We are motivated by this belief that digital archaeology is defined by the productive, pedagogical fail. It is this aspect of digital archaeology that also makes it a kind of public archaeology, and that failing in public can be the most powerful thing that digital archaeology offers the wider field.
We implore you to do your research so that others can retrace your steps; even a partial step forward is a step forward! When you find a colleague struggling, give positive and meaningful feedback. Be open about your own struggles, but get validation of your skills if necessary. Build things that make you feel good about your work into your work.
Time is your enemy - bit rot and link rot - deprecated formats - disappearing web resources and problems with the “cloud” - lost institutional knowledge and unconscious assumptions
-
Tools - metadata creation - fixity
+
With pre-planning and thought directed at topics like format, structure, and descriptions, you can do a lot to extend the life of your work. From the outset, consider whether you intend to prepare your data for ultimate disposition in a digital repository, maintain it yourself, or both. In the discipline of digital preservation, the LOCKSS (Lots of Copies Keep Stuff Safe) principle recommends storing data in at least three separate locations to ensure redundancy.
+
For a deeper dive into digital preservation, see Trevor Owens, The Theory and Craft of Digital Preservation, Johns Hopkins University Press, 2018 (full open access preprint available).
+
+
+
An archival box full of legacy media in the section author’s workspace, waiting to be transferred to stable formats.
+
+
Time is your enemy Archaeologists are well aware that excavation is a destructive act. Conservation is never forever. When we dig, we participate in a process of rearranging physical material in ways that make it both easier to interpret and more vulnerable to inevitable degradation. In a way, the same can be said about archaeological data. Here are some common risks:
+
+
deprecated formats
+
bit rot and link rot
+
disappearing web resources and problems with the “cloud”
+
lost institutional knowledge and unconscious assumptions
+
create thorough metadata as you go. This will help you, those who use your data in the future, and those who are tasked with curating your datasets.Structural metadata and descriptive metadata. Different purposes but both can increase durability of data.
+
+
Tools - metadata creation - fixity or other checksum tools
Xu, Weijia, et al. A Portable Strategy for Preserving Web Applications Functionality. IEEE, 2017, pp. 1–4. CrossRef, doi:10.1109/JCDL.2017.7991578. [Jolene can’t access this]
Wang, Haiyan, Zhongshi He, Yongwen Huang, Dingding Chen, and Zexun Zhou. 2017. “Bodhisattva Head Images Modeling Style Recognition of Dazu Rock Carvings Based on Deep Convolutional Network.” Journal of Cultural Heritage 27: 60–71. doi:https://doi.org/10.1016/j.culher.2017.03.006.
Digital archaeology is necessarily a public archaeology. This is its principal difference with what has come before, for never forget, there has been at least a half-century of innovative use of computational power for archaeological knowledge building.
Geospatial, digital and Web-based tools are now central to carrying out archaeological research and to communicating archaeological information in a globalized world. Until recently, the accumulation and effective management of digital archaeological data have been the primary focus of archaeologists (Evans and Daly 2006). Under this model, scholars emphasize the ‘integration’ into archaeology of computing technologies, and how, by utilizing current available computing memory and processer speed, one does archaeology, only better (Daly and Evans 2006: 1). This situation in turn demonstrates the ‘marriage between the two’, archaeology and computing (Daly and Evans 2006: 2).
For Evans and Daly (2006), writing in the first decade of the 21st century, digital archaeology was synonymous with the use of Information and Communication Technology or ICT, and reflected wider efforts at that moment to transform education through newly available digital tools. Some scholars and policy makers believed that digital technologies were the answer to pressing global social issues such as poverty, a point that we will discuss later.
-
More recently, in his inaugural editorial for the open-access journal, Frontiers in Digital Humanities, (Costopoulos 2016) argues that ‘digital archaeology has been [here] a while’. Computing in archaeology, that is ‘doing archaeology digitally’ as Costopolous remarks, constitutes a ‘state of normality’ in archaeological practice. This view places emphasis on the availability of digital tools and their use in institutional contexts, overlooking the highly structured nature of social groups that employ these tools, and where, how and why these technologies are created and used. While fruitful, these views tend to obscure broader developments in the social sciences and humanities, of which archaeology is a part, and underestimate the changing relationship between archaeology and society.
+
More recently, in his inaugural editorial for the open-access journal, Frontiers in Digital Humanities, Costopoulos (2016) argues that ‘digital archaeology has been [here] a while’. Computing in archaeology, that is ‘doing archaeology digitally’ as Costopolous remarks, constitutes a ‘state of normality’ in archaeological practice. This view places emphasis on the availability of digital tools and their use in institutional contexts, overlooking the highly structured nature of social groups that employ these tools, and where, how and why these technologies are created and used. While fruitful, these views tend to obscure broader developments in the social sciences and humanities, of which archaeology is a part, and underestimate the changing relationship between archaeology and society.
1.1.1 A distant view
Ethan Watrall has drawn the history of computational archaeology/digital archaeology all the way back to the pioneering work of James Deetz in the 1960s, who used computers at MIT to perform stylistic analyses of Arikara ceramics (Ethan Watrall 2017, Deetz (1965)). Most early interest in computation for archaeology was centred on the potential for computational databases, although ambition often out-stripped capability. By the 1970s, serious efforts were being put into work to build the infrastructural knowledge necessary to make and usefully query archaeological datasets. One can see this concern play out by considering a topic model(Shawn Graham 2014) of the early volumes of the Computer Applications in Archaeology (a topic model is a way of deducing latent patterns of discourse within text, based on patternings of words (See Graham, Weingart, and Milligan 2012)):
@@ -321,7 +314,7 @@
1.1.6 Exercises
Find the markdown file on your machine that you created with Dillinger. You can drag-and-drop this onto the list of files in your repository; Github will know that you want to upload it. OR you can click on the ‘upload files’ button, and then ‘select files’ and click on the file that way (much like adding an attachment to an email).
Github will upload the file; you can upload more files at this point, or click on the green ‘commit’ button at the bottom.
-
** As you work through this book, we encourage you to write your thoughts, observations, or results in simple text files.** This is good practice whether or not you embark on a full-blown digital project, because ultimately, if you use a computer in your research, you have gone digital.
+
As you work through this book, we encourage you to write your thoughts, observations, or results in simple text files. This is good practice whether or not you embark on a full-blown digital project, because ultimately, if you use a computer in your research, you have gone digital.
Each section in this book is broken down into an overview or discussion of the main concepts, and then followed up with skill-building exercises. The computational environment - provided via a Jupyter notebook- is running on someone else’s servers. When you close the browser, it shuts down.
+
Warning! The computational notebooks that we provide that are running in the [binder][http://mybinder.org] environment will time out after ten minutes’ inactivity.
+
Your work can be saved to your own machine and reloaded into the environment later, or you can ‘push’ your changes to your own online repository of work (to learn how to push work to a Github repository, see the sections on Github & Version Control and Open Notebook Research & Scholarly Communication so you’ll be able to get your work and data out of the Jupyter Notebooks and onto space that you control). The best way to use this book is to make sure you have at least one hour blocked out to read through a section, and then two hours to go through the section again if you’re working on the exercises. We find that the work goes better if those blocks are interspersed with breaks every twenty minutes or so.
+
Do you notice that stripe down the side of the screen at the right? That’s a tool-bar for annotating the text, using a tool called Hypothes.is. If you highlight any text (go ahead, highlight that phrase right now by right-clicking and dragging your mouse!) a little pop-up will ask you if you want to annotate or highlight the text. If you choose annotate, a writing pane will open on the right. Using the Hypothesis tool requires a reader to create a login and account with Hypothesis, which is managed by the Hypothesis site, not us.
+
By default, such annotations are made public. Private annotations can only be viewed by the particular individual who made them. All annotations (both public and private) have their own unique URL and can be collated in various ways using the Hypothesis API (here’s an example). Please tag your annotation with odate to allow easy curating of the public annotations.
+
Please note that any public annotations can be read by any other reader. These can also be responded to, as well - which might make a great classroom activity! A class can create group annotations which are only visible to participants in that group (instructions here). Annotation is a tool for research; personal reaction to anything we’ve written in ODATE should be done via the reader’s blog while leaving an annotation on ODATE linking to the blog piece. Because the API or ‘application programming interface’ for Hypothesis allows one to retrieve annotations programmatically, there is a growing world of scripts and plugins for managing or retrieving those annotations. Kris Shaffer has created a Wordpress plugin to pull annotations to a new Wordpress post (details are linked here), which might be another great option for a class blog working through ODATE.
+
Over time, the parts of the text that are heavily annotated will look as if someone has gone over them with a yellow highlighter. You can use this to help guide your reading - perhaps that’s a part where many people had problems, or perhaps it’s a part that sparked a really interesting discussion! Group annotation like this promotes ‘active reading’, which means that you’re more likely to retain the discussion.
+
Finally, if you’d rather not read this as a web page, you can grab a pdf copy by pressing the download button above (the downwards-facing arrow icon) and printing out just the bits you want or need. If you’d rather read this text via an e-reader or iBooks or similar, the download link will also give you an ePub version. Individuals who use a screenreader or other assistive device might prefer to work with the pdf or epub versions. Please do let us know if there is any way we can make this text more accessible for users with particular needs. Since this text is fundamentally a series of plain-text files that we then manipulate to create these different outputs, it should be straightforward for us to adapt accordingly!
Digital archaeology as a field rests upon the creative use of primarily open-source and/or open-access materials to archive, reuse, visualize, analyze and communicate archaeological data. This reliance on open-source and open-access is a political stance that emerges in opposition to archaeology’s past complicity in colonial enterprises and scholarship; digital archaeology resists the digital neo-colonialism of Google, Facebook, and similar tech giants that typically promote disciplinary silos and closed data repositories. Specifically, digital archaeology encourages innovative, reflective, and critical use of open access data and the development of digital tools that facilitate linkages and analysis across varied digital sources.
-
To that end, this document you are reading is integrated with a cloud-based digital exploratory laboratory of multiple cloud-computing tools with teaching materials that instructors will be able to use ‘out-of-the-box’ with a single click, or to remix as circumstances dictate. Part of our inspiration comes from the ‘DHBox’ project from CUNY (City University of New York, (link), a project that is creating a ‘digital humanities laboratory’ in the cloud. While the tools of the digital humanities are congruent with those of digital archaeology, they are typically configured to work with texts rather than material culture in which archaeologists specialise. The second inspiration is the open-access guide ‘The Programming Historian’, which is a series of how-tos and tutorials (link) pitched at historians confronting digital sources for the first time. A key challenge scholars face in carrying out novel digital analysis is how to install or configure software; each ‘Programming Historian’ tutorial therefore explains in length and in detail how to configure software. The present e-textbook merges the best of both approaches to create a singular experience for instructors and students: a one-click digital laboratory approach, where installation of materials is not an issue, and with carefully designed tutorials and lessons on theory and practice in digital archaeology.
+
To that end, this document you are reading is integrated with live open code notebooks that can be re-used, altered, or extended. Part of our inspiration comes from the ‘DHBox’ project from CUNY (City University of New York, (link), a project that is creating a ‘digital humanities laboratory’ in the cloud. While the tools of the digital humanities are congruent with those of digital archaeology, they are typically configured to work with texts rather than material culture in which archaeologists specialise. The second inspiration is the open-access guide ‘The Programming Historian’, which is a series of how-tos and tutorials (link) pitched at historians confronting digital sources for the first time. A key challenge scholars face in carrying out novel digital analysis is how to install or configure software; each ‘Programming Historian’ tutorial therefore explains in length and in detail how to configure software. The present e-textbook merges the best of both approaches to create a singular experience for instructors and students: a one-click digital laboratory approach, where installation of materials is not an issue, and with carefully designed tutorials and lessons on theory and practice in digital archaeology.
This is not a textbook about learning how to code. Rather, it is about instilling the habits of thought that will enable success when confronted with digital novelty, the habits of thought that will enable you to determine how to work with digital materials, and the habits of thought that permit you to see where and when digital approaches will make the difference in your research. Skills change; techniques evolve; new tools emerge. Habits of thought are hard to cultivate but have staying power!
Through this textbook, we aim to offer a learners’-perspective-view on digital methods in archaeology, that is, how we might think with, and through, digital sources of information, digital tools and technologies and their relationship with society. We are deeply aware of how rapidly both digital sources and technologies can change, particularly on the Web; we therefore present this e-textbook and open-learning environment as a guide to best practices when working with available digital data and digital tools, what kinds of analysis are possible, how to perform these analytical techniques, and how you might publish your data, making them re-usable for another scholar and ethics and ethical issues in doing digital archaeology.
+
We have not elected to try to cover every possible topic. By design, this book is meant to grow, branch, and change with time. It is meant to foster uncoverage and be used to supplement or complement an instructor’s own situated approach. Annotate the text using the Hypothes.is. Take it to bits. Use and adapt the parts that make most sense in your own particular learning context.