IIIF Curation Platform: Recent Developments to Support Large Language Models and the Latest APIs

The IIIF Curation Platform (ICP) introduced the concept of “curation” for user-driven content creation. Since its start in 2016, we have reported its development progress in IIIF conferences, which resulted in nine open-source software tools designed for various applications. This presentation introduces our recent developments to the ICP to support large language models (LLM) and the latest APIs.The first topic is IIIF Tsukushi Viewer, which supports LLM-assisted reading of the IIIF materials. The current viewer is designed to assist in reading historical (classical) Japanese books written in cursive characters (Kuzushiji) with old grammar and vocabulary. Most Japanese people cannot read those books, but a user can roughly understand what is written in the book by asking LLMs to summarize, translate, or answer prompts. We already deployed the service for about 3000 books, 600,000 pages, with text transcribed by Kuzushiji OCR. Technically speaking, the viewer communicates with Tsukushi Content Service and Tsukushi Chat Service. Tsukushi Content Service stores the OCR or transcribed text for each canvas, and the viewer can access the REST API with Manifest URI and Canvas ID parameters to get the text of each canvas. Tsukushi Chat Service offers several methods for the LLM-based chat service, and the viewer can access the JSON-RPC API with context and prompt parameters to get an LLM response to each prompt. IIIF Tsukushi Viewer first calls Tsukushi Content Service to collect the text of user-specified canvases for the chat context. The viewer then calls Tsukushi Chat Service with the chat context and a user-given prompt. It also supports RAG-style chat by vector similarity using PostgreSQL as a backend database and pgvector as a vector search engine.The second topic is updating the ICP to support the IIIF Image/Presentation API 3. Because ICP was initially designed for API 2 (v2), adding support for API 3 (v3) was challenging. To avoid the “second-system syndrome” by refactoring everything, we introduced a compatibility layer to convert v3 to v2 internally. We check compliance with v3 based on the IIIF cookbook and continue enhancing support for v3.

Citation

Asanobu KITAMOTO, Jun HOMMA, "IIIF Curation Platform: Recent Developments to Support Large Language Models and the Latest APIs", 2024 IIIF Online Meeting, 2024-11

Related Resources and Related Websites

Related Pages in this Site

| Link 1 |