Promoting Professional Development and Best Practice in EE
Research Summary Mapping
As part of the discovery process for the NAAEE, C&NN, and Pisces research portal/database project, Tamarack Media Cooperative has undertaken a simple data mapping and analysis excercise of the existing content provided by each partner. The goals for this mapping/analysis include:
- Identify extent of overlapping content between two research databases.
- Identify gaps in summarized research to guide future summary writing.
- Identify any technical or editorial issues related to content feeds and synching.
This webpage reflects some initial analysis with embedded interactive visualizations. Given the limited budget for this exercise, only the most basic data clean up and quality assurance process has been undertaken so there may be some anomalies and mischaracterizations in the visualizations below.
Overview of Existing Research Summaries
The two sources of seed content for the research portal are:
- Children & Nature Network's Research Database exported 6/21/17 which includes 627 summarized articles (82 of which are not yet posted on their website).
- The Environmental Education Research Bulletin series produced by NAAEE, Stanford and ChangeScale which summarized 227 articles over nine issues. An additional three issues are in progress, which will include another 65 summaries, bringing the total to 292. Additional research related content has been posted to NAAEE's professional development platform eePRO, but only EERB reflects a curated summary process parallel to the C&NN database.
- The combined seed content for the portal totals 919 summarized articles, putting a target of 1,000 summaries within reach for early 2018.
While EERB covers a specific number of journals more focused on environmental education, the C&NN editorial process casts a much wider net, as reflected by the bar graph below (scroll to the right to see the "long tail" of the journals).
Date of Publication
The EERB cover a more limited date range, with each issue covering a six month period. The graph below captures the longer time period captured by the C&NN database as well as recent growth in the volume of summaries (including many pending summaries).
Note: After noticing a surpising number of older pending articles for C&NN, I discovered that many of these appear to be duplicate articles in the export classified as both pending and published (perhaps different versions of a summary).
As a very simple content analysis, a word cloud was created based on the titles of articles appearing in each database (using the 1,000 most frequent words in each).
Mapping of Taxonomies Used to Categorize Articles
C&NN has a sophisticated system of categorization used to facilitate searching the research database with a specific focus on nature engagement. NAAEE uses a series of taxonomies for categorizing research (and other content) in the professional development community eePRO. The taxonomies used at informalscience.org include detailed categorization of resource type with many options for informal learning environment. In addition, we need to incorporate the Pisces/EGA broad categories of grantmaking: Education, Conservation, Environmental Justice, Youth Development, Civic Engagement.
The embedded Google spreadsheet below includes the full metadata options as well as some initial mapping of the NAAEE and CAISE terms to the C&NN taxonomies. Some initial reactions:
- We need to decide a broad approach. Do we try to capture as much information as possible on the portal (even if it isn't captured at the same level for all content from all sources) or do we go with a lowest common demoninator (to ensure that all content is tagged as consistently and meaningfully as possible)?
- The taxonomies for C&NN and CAISE are built upon some assumptions (the research in question is about nature engagement, the setting of the learning is informal, etc) which don't translate to the more generalized case of all EE research. But, assuming the research is related to EE, what are the natural ways of categorizing research that should be considered?
- For the NAAEE summary-writing process - primarily through Stanford's EERB, the communication tools for eeWORKS, and the portal content created by Duke - what level of metadata coding is desireable? For example, the taxonomies related to methods, discipline, and demographics could be useful additions to the portal, but capturing this information for each summary in a consistent manner would be a new component of the summary-writing process.
- The Pisces/EGA taxonomy is a requirement but it isn't obvious the best way to incorporate into existing tagging or how to use consistently. For example, for EERB content, all will be related to Education and yet only a small number will relate to any of the other terms - does that mean there should be a primary categorization and secondary categorization, or is it ok to check all that apply? The EGA field could stand alone, or it could be embedded in an existing field, for example restructuring the "Topics" taxonomy to be organized with the EGA terms as top level parent terms and the existing terms of child terms.
Overlap Between Research Databases
By compiling all citations into a single spreadsheet and sorting by journal, publication year, and title, nine duplicate summaries were identified, which translates to approximately 1% of the total number of combined summaries.
Kalvaitis, D., & Monhardt, R. M. (2012). The architecture of children’s relationships with nature: a phenomenographic investigation seen through drawings and written narratives of elementary students. Environmental Education Research, 18(2), 209-227.
Morag, O., & Tal, T. (2012). Assessing learning in the outdoors with the field trip in natural environments (FiNE) framework. International Journal of Science Education, 34(5), 745-777.
Liefländer, A. K., Fröhlich, G., Bogner, F. X., & Schultz, P. W. (2013). Promoting connectedness with nature through environmental education. Environmental Education Research, 19(3), 370-384.
Damerell, P., Howe, C., & Milner-Gulland, E. J. (2013). Child-orientated environmental education influences adult knowledge and household behaviour. Environmental Research Letters, 8(1), 015016.
Zimmerman, H. T., & McClain, L. R. (2014). Intergenerational learning at a nature center: families using prior experiences and participation frameworks to understand raptors. Environmental education research, 20(2), 177-201.
Including the pending summaries of EERB and C&NN reveals a further nine duplicates in 2016, perhaps reflecting more curational overlap as the C&NN dataset has expanded.
Kacoroski, J., Liddicoat, K. R., & Kerlin, S. (2016). Children's use of iPads in outdoor environmental education programs. Applied Environmental Education & Communication, 15(4), 301-311.
Monroe, M. C., Ballard, H. L., Oxarart, A., Sturtevant, V. E., Jakes, P. J., & Evans, E. R. (2016). Agencies, educators, communities and wildfire: partnerships to enhance environmental education for youth. Environmental Education Research, 22(8), 1098-1114.
Wight, R. A., Kloos, H., Maltbie, C. V., & Carr, V. W. (2016). Can playscapes promote early childhood inquiry towards environmentally responsible behaviors? An exploratory study. Environmental Education Research, 22(4), 518-537.
Waite, S., Bølling, M., & Bentsen, P. (2016). Comparing apples and pears?: a conceptual framework for understanding forms of outdoor learning through comparison of English Forest Schools and Danish udeskole. Environmental Education Research, 22(6), 868-892.
Williams, C. C., & Chawla, L. (2016). Environmental identity formation in nonformal environmental education programs. Environmental Education Research, 22(7), 978-1001.
Wilks, L., & Harris, N. (2016). Examining the conflict and interconnectedness of young people’s ideas about environmental issues, responsibility and action. Environmental Education Research, 22(5), 683-696.
Dieser, O., & Bogner, F. X. (2016). Young people’s cognitive achievement as fostered by hands-on-centred environmental education. Environmental Education Research, 22(7), 943-957.
Greenwood, A., & Gatersleben, B. (2016). Let's go outside! Environmental restoration amongst adolescents and the impact of friends and phones. Journal of Environmental Psychology, 48, 131-139.
Sonti, N. F., Campbell, L. K., Johnson, M. L., & Daftary-Steel, S. (2016). Long-term outcomes of an urban farming internship program. Journal of Experiential Education, 39(3), 269-287.
Gaps in Research Summaries
The team at Duke is undertaking both content curation and writing of new summaries. A major component of their content strategy involves extracting, adapting, and writing summaries for articles that have been included in compilations of essential reading and critical concepts for the field. This will address to some extent the limitation of date range identified above.
This analysis is not sophisticated enough to identify any specific topic/issue areas, research methodologies, or other dimensions of the research database that may be lacking. Hopefully this mapping exercise is enough to get this kind of conversation started among the relevant stakeholders.
Technical/Editorial Consideration for Content Feeds/Synching
The following notes based on reviewing existing summaries should inform the development phase:
- While a small number of overlap exists between existing summaries in the respective databases, this does mean we need to account for identifying duplicate content in the feeds of new and updated research.
- The easiest way to identify an article for synching and import would be via a unique identifier like a DOI, however only 93% of the EERB articles (compiled in Zotero) and 82% of the C&NN articles had an associated DOI. Therefore a combination of fields or another database specific unique identifier might be required for content feeds and synching updates.
- Reviewing some notes from the DC meeting, I realized Alan referenced getting DOI's for content that currently doesn't have a DOI, including perhaps the research summary or synthesis (https://www.doi.org/faq.html). This could be incorporated into our content creation process to ensure we are using consistent, unique identifiers.
- The most logical additional fields for identification would be related to the journal, date/volume of publication, title, and authors. However, each of these values may have some inconsistencies between the datasets, leading to duplicates not being captured. For example:
- When looking at identical articles, the titles were occasionally formatted differently, for example capitalization.
- Authors were formatted slightly differently in the exported spreadsheets. Author mapping didn't promise to reveal much useful information, so this wasn't an issue in the mapping exercise, but in Biblio author duplicates/merging will be an ongoing process.
- There were a few slight variation of journal names, for example omission of leading "The" or swapping of "&" and "and".
- To allow for updates of content from an external source, we will likely need to save the source ID.
- Exports via Excel resulted in strange character handling requiring some basic clean up for the analysis - the feeds of content should avoid this, but special characters may require additional attention.