Why online platform dLibrary.org was created and its advantages for archivists, librarians and researchers (Kirill Fesenko, Aug 2015)

Why this platform was created

The founders of dLibrary.org have been digitizing and putting collections online for access since the early 90’s. Working with archives and libraries in various settings, we encountered one common problem --  lack of software which is easy to use for a wide range of collections and materials. Popular digital library solutions are not fitting well archival needs and vice versa; free open source programs are not flexible or scalable enough and often lack adequate tech support; and internal development of software requires hiring programmers and maintaining expensive operations. Lack of affordable and user-friendly software is one of the main reasons curators still keep wealths of finding aids and digitized primary sources on hard drives and CD-ROMs. So we decided to focus efforts on the development of a “dream platform” for archivists, librarians and researchers. 

Platform architecture and user interface

In laying out the architecture and user interface for the platform, we relied on experience gained from our past digital archive/library projects (completed projects), research initiatives (Extending the Reach of Southern Sources. Proceeding to Large-Scale Digitization of Manuscript Collections (2009)) as well as feedback from hundreds of scholars, archivists and librarians worldwide with whom we communicated over the years. The platform’s interface incorporates ideas based on works of Thomas Mann, a reference librarian at the Library of Congress, dedicated to library research models and true browse displays for online systems. 

The platform also went through extensive testing and received high evaluation at a number of institutions, including:

- Brill Publishers (Netherlands)

- Centre d’études des mondes russe, caucasien et centre-européen (Cercec) (France)

- Davis Center for Russian and Eurasian Studies at Harvard University (U.S.)

- Department of Historical Information Science, Moscow State University (Russia)

- East View Information Services (U.S.)

- Faculty of History, Higher School of Economics, National Research University (Russia)

- German Historical Institute in Moscow (Germany)

- International Institute of Social History (Netherlands)

- Italian Society for the Study of Modern and Contemporary History (SISSCO) (Italy)

- Russian State Archive of Socio-Political History (RGASPI) (Russia)

- Slavic and East European Collections, Stanford University Libraries (U.S.)

- State Historical Public Library of Russia (GPIB) (Russia)

- Stockholm School of Economics (Sweden)

- Yale University Press (U.S.).

Use cases

The first pilot version of the platform became available for testing in 2009. Since then, the platform has undergone three major revisions (current version is 4.8.22). To date, over 5 million pages of archival and library materials were published on the platform for public access. Publishers are using the platform to host and provide access to complete digital archives of publications. 

The State Historic Public Library (SHPL) uses the platform for daily value-added online publishing of collections for public access. The digitization/publishing process is organically built into the library's daily operations assuring continuous addition of freshly digitized materials to a growing number of online collections. Library readers are regularly updated about new online titles on the library's web and social sites (Facebook, V Kontakte, LiveJournal). The readers use social sites to comment on the currently added titles and provide ideas for digitization of new material. The readers also enjoy online access to incoming "uprocessed" titles without descriptions and can monitor their publishing progress in real time.

SHPL librarians also use the platform for value-added processing of books which involves manual tagging and description of portreits, maps and schemas found on the books' pages and inserts. As result, readers enjoy deeper access to contents of books and can browse new large collectoins of historical illustrations.

Besides stress-free publication and management of large online archival and library collections, the platform is used for collaborative description of primary sources and research by groups of remote archivists and scholars. The German Documents in Russia project, for example, scanned in bulk previously inaccessible large collection of German WWII archives with minimal descriptions. As documents were scanned, they were also uploaded to the online platform for remote study by historians. The group researched the documents for new finds and added descriptions on the folder level. Upon completion of this work, the collection was released for public online access earlier this year. 

Currently, a new project aimed to provide online access to dispersed psychology and special education research archives is in active scanning and online publishing stage. The project uses the platform as an aggregation system where a number of scanned “hidden archives” are brought together for collaborative online description and study by a group of researchers from several institutions. The project stakeholders plan to release some of these psychology archives to public access eventually, while others will be available to registered users only with password/IP address access. 

Platform advantages for archives, libraries and researchers

The platform users find attractive that its interface is intuitive. It is also expandable for materials and projects of various types, and can be employed in ways that fit organizational needs best today and in the foreseeable future. 

• No programming or other technical expertise required: archive/library core staff can create, finetune and manage large online collections using administrator toolbox;

• Supports high volume publication of material in various formats (over 3,000 pages/daily), including archival documents, books, journals and newspapers, maps, photographic collections, etc. The tiling module allows high degree of zooming to scanned maps;

• Customizable interface for work with large collections: all major interface elements can be fine-tuned using admin tools, including display of documents, description fields and browse displays for indexes to collections. The DocView browser is optimized to display and zoom in/out quickly on hundreds and thousands of document images providing scholars with comprehensive (bird’s-eye) visual overview of large collections. 

• Creation of hierarchies (fond-series-folder-document, etc.) required to mirror hierarchical organization of physical collections -- important feature for archival collections which is commonly underdeveloped in traditional digital library software which is oriented towards “flatter” book collections;

• Creation of collection description, metadata profiles and data entry templates. The collections can be fine-tuned for organization-specific descriptions, bibliographic and metadata standards. Custom data entry templates help streamline description of collections and documents. Platform includes developed module for metadata type settings and editing.

• Full text search for records and OCRed text. The full-text search engine allows quick searches with found keywords highlighted in the context of a document or its description. The advanced search handles more complex queries for individual fields of descriptions and their combinations.

• Collection access control. The administrative module allows multi-tiered access to records and documents to satisfy specific requirements of any collection. Access can be restricted by username/password or IP address to documents in high resolution only (descriptions and small size document page icons will be available to public) or to any parts of description and full image documents.

• Usage statistics module allows monitoring and reporting of each collection’s usage.

• Built-in data backup solution on physically separate server (Dark archive).

• Supports new methods of collaborative online organization, description and research of large archival collections. 

• Platform subscription plan can be flexibly adjusted in accordance with organizational budgets and development plans for digital collections. Each plans offers full platform functionality and administrative control over the online collections, as well as regular version updates and tech support.