One of our developers' review "this conference was surprisingly good" - this is a HUGE compliment #dlfforum @clirnews.bsky.social
We're the last session so you could either get a jump on your trip to the airport or come to our session.
Lauren and I will make it worth your while 👿 #dlfforum
Not suitable for this task without lots of review and validation. Not a time saver. Low confidence from tool gave higher user confidence in answer. #DLFForum #DLF2025
Strengths - feature identification pretty good, general geographic identification ok. Opacity a real issue in reviewing results. Also verbose. Inconsistency on prompted task, and time consuming for people to review #DLFForum #DLF2025 At one point ChatGPT asked if it could do something else LOL
Prompt drift an issue with ChatGPT or would start referencing previous images. Prompt drift not an issue with other models #DLFForum #DLF2025 Aerial photos in sequence - models could recognize that but unable to judge the set as a group or course correct.
Claude would give broad answers that were more correct, other models would confidently mislabel images. #DLFForum #DLF2025
Chat CPT performed similarly for high context and low context prompt. Differences seen in other models. ChatGPT also stopped giving confidence scores #DLFForum #DLF2025
ChatGPT overall did the best, followed by Gemini, then Claude according to scoring system developed for the project. Confidence assigned by models - ChatGPT fairly high, Claude low. #DLFForum #DLF2025 - Claude confidence and correctness in alignment, ChatGPT and Gemini not in alignment.
Prompting issues - phrasing, context, specificity, and user role all important. Tried low context and high context prompts. Did not give extra guidance. #DLFForum #DLF2025
Fed LLM sequence of images and asked for geographic info like latitude and longitude coordinates, placenames. - mountain landscapes, aerial images - 20 samples of each #DLFForum #DLF2025
Used VPN to attempt to prevent LLM abilities to detect location by user query. Also created burner emails / new user accounts for all of these platforms. Not sure if this was effective, but an attempt was made!
#DLFForum #DLF2025
University of Colorado Boulder looking at three free models to see how they did with geographic resources like aerial maps. Wanted to use DeepSeek but it didn't do image analysis
#DLFForum #DLF2025
In session on Measuring and Critiquing genAI tools for Geographic Metadata Creation
#DLFForum #DLF2025
Responsible AI in Libraries and Archives team has released our interactive Viewfinder toolkit.
Toolkit (interactive website): www.lib.montana.edu/responsible-...
Project info: www.lib.montana.edu/responsible-...
#DLFForum #DLF2025
Reasoning with Small Language Models (SLM) to Create Trustworthy GenAI
Slides: doi.org/10.5281/zeno...
System Prompt: gist.github.com/jasonclark/1...
Model Context Protocol server:
gist.github.com/jasonclark/4...
#DLFForum #DLF2025
Important to keep context in mind when doing this type of assessment. What relationships to feature when describing a family photo. Example of inconsistent terms over time - farmer vs agricultural laborer with racial connotations. #DLFForum #DLF2025
Overview of process using lexicons and csv metadata with MaRMAT. Example of spreadsheet identifying aggrandizement terms like "pioneer" "gentleman" "famour" #DLFForum #DLF2025
Work on using MaRMAT for assessment of collections at University of Louisville. Note that @rjwcore.bsky.social has stickers!!! #DLFForum #DLF2025
Development of working group for harmful content statement at UNR had broad representation - 10 members across different departments, working on it for 6-8 months. #DLFForum #DLF2025 statement used to propel practical work such as work on Basque posters and local vocabulary
Harmful content statement elements - accessibility of statement to the public, scope of materials, acknowledgement of harm in a collection + metadata describing it, statement of future action/goals #DLFForum #DLF2025
Data Advocacy for All Toolkit da4all.github.io #DLFForum
Starting out by defining harmful, problematic, and outdated content - remember nuance between these definitions. #DLFForum #DLF2025
Hope to develop tool based on feedback and serving different use cases - community exists. Importance of social infrastructure for continued development of DH software. #DLFForum #DLF2025 Developers important but community also contributes use cases, feedback, etc. IIIF as a standard also key
How to work with spatial metadata at scale, how to deal with interoperability in IIF ecosystem, standardized spatial metadata. Termination of grant was 19 months into 3 year grant period. They are continuing to do work, but it is stalled without $$ for development #DLFForum #DLF2025
Mention of terminated NEH grant for development of Allmaps curator project :( - how to manage the annotations and ensure accuracy of the work.
#DLFForum #DLF2025