Florida Online Journals (OJ): Harvesting and Indexing

Use Journal Sections to separate out different types of content.

• Background:
  • Most harvesters will wish to harvest articles.  Most harvesters will not want to harvest tables of contents, front matter, and back matter.  Florida OJ allows harvesting one journal section at a time.  (This is using the OAI-PMH setSpec query.  In Florida OJ, each journal has it's own OAI-PMH repository - so journal content is automatically separated from other journals.  Then, within each journal, each journal section is an OAI-PMH set.  Sets can be harvested one-at-a-time.)
  • A recommended best practice is to set up journal sections, then separate out front matter, back matter, and full issues into separate sections.  Articles can be grouped together or can be divided among several sections.  In this way, all sections containing articles can be harvested while sections containing administrative material, front matter, etc. can be ignored.  The key point is to separate articles from other kinds of content - separate articles from not-articles.
  • Journal sections can also be used to separate peer reviewed from non-peer reviewed content, for example, for inclusion in indexers where peer review is required.
  • Additionally, most harvesters will not want repetitive records - ie. several issues with identical titles on the records.  OJS 3 has an area for uploading "Issue PDF".  If you are loading new content, it is recommended to load full issue PDFs to the "Issue PDF" area of Florida OJ.  If you have back issues uploaded as articles, then it is recommended to make a journal section for full issues, and sort the full issue PDFs to that journal section.
• Instructions:
  • To quickly see what journal sections exist, and to add new journal sections:  Log in to your journal, from the journal logged in view click to "Settings" "Journal" then the tab for "Sections".
  • To get a report of what section each article on the site is loaded to:  Log into your journal, from the logged in view click to "Tools" "Statistics" then pull the "Articles Report".
  • To change what section an article is in:  Log into your journal, from the logged in view click to "Issues" and locate the issue that the article is in.  Click the blue triangle next to the issue, and then click the "edit" button.  This will open a menu showing all articles in the issue.  Click the blue triangle next to the article you wish to edit, then click the "Submission" button.  In the top right corner, click to "Metadata".  Use the drop down menu for "Section" to select the section you want to move that article to, and click "Save" at the bottom of the form to apply the changes.

Review instructions from the search that will use your records, and check required fields.

• Background:
  • Commonly, any indexing services which collect records will have a minimum set of information they need (ie. title, author, date, abstract, etc.).  Review guidelines or instructions from the indexing service, and then assess your journal's records to see whether you meet the guidelines.
• Instructions:
  • To assess metadata:  Log into your journal, from the logged in view click to "Tools" "Statistics" then pull the "Articles Report".  This will show:  title, abstract, author names, language of the article, and journal section.
  • To assess metadata:  To assess fields like keywords, run an OAI-PMH harvest with MARCEDIT, then use Open Refine to assess.  See Instruction sheet and video .  This will allow you to assess all fields.  You have to harvest in oai_dc , because Open Refine does not work well with Florida OJ.
  • For assessing metadata beyond what is available through the Articles Report, it is recommended to contact help@flvc.org with "Florida OJ" in the subject line and request assistance in getting started.
  • To edit metadata for an article:   Log into your journal, from the logged in view click to "Issues" and locate the issue that the article is in.  Click the blue triangle next to the issue, and then click the "edit" button.  This will open a menu showing all articles in the issue.  Click the blue triangle next to the article you wish to edit, then click the "Submission" button.  In the top right corner, click to "Metadata".  Make the necessary edits, and click "Save" at the bottom of the form to apply the changes. 
  • If you do not see a specific field while editing metadata, then check the following to ensure that the field is enabled for your journal and for the journal section containing the article:  Check in "Settings" "Workflow" then look for the field under the heading "Submission Metadata".  Some fields, like keywords, have to be enabled in order to edit or add them.

Submitting Florida OJ records to EBSCO Discovery Services (EDS):

*Note* Out of public colleges and universities currently using Digital Services and Open Educational Resources services, the only institutions contributing records to EDS are institutions using EDS for resource discovery on their campuses.  Although digital library resources are open access, at this time, it is customary for EDS to configure each campus's search to access only resources contributed by that campus.  Therefor, this guide is written with the goal of providing information to provide to EDS, if you are using EDS for resource discovery on your campus and wish to have your Florida OJ records show up in your campus's EDS instance.

Guidance for the EDS questionairre

General Questions:

• Because each journal in Florida OJ has a separate OAI-PMH feed, you will need to fill out a separate questionairre for each journal liaised through your campus.
• Name of the database:  It is suggested that you use the journal's name.
• Description of the database:  It is suggested that you use the short description provided on Florida OJ's homepage.  This is written by the journal and is accessible from the Journal Manager, Setup area.
• Type of database:  It is suggested to use Institutional Repository.  Published journal articles are most similar to institutional repository content.
• IR Software/Vendor:  Open Journal Systems (OJS).

Data Acquisition Questions:

• Metadata standards used:  MARCXML, RFC1807, Dublin Core.  (Note:  OJS software also supports the National Library of Medicine (NLM) metadata format, however this is not available for most journals on the site.  At this time, FALSC believes that this is because required fields are not in the journal metadata.  For example, Expiration date, the date the resource is to be reviewed or pulled, is a required field in NLM, but is not present in most article metadata in Florida OJ.  In short, do not specify NLM on the EDS questionairre.)
• Files to be included:  Recommended to put "All".  The only exception is when you have both a full issue PDF, and individual article PDFs.  In that case, verify that the full issue PDFs are consistently in a separate journal section in Florida OJ, and specify not to include them.  Please contact help@flvc.org with "Florida OJ" in the subject line, if you have any questions about this.
• Is the OAI accessible:  Yes.
• If not, can you FTP records:  Not applicable. (However, you also can export XML records in METS directly from Florida OJ.  This should not matter for EDS, but may matter if you are looking to contribute to specialty databases, for example, CABI.)
• Would you like to have your data harvested on a schedule:  Recommended to put the publication schedule of your journal.  For example, quarterly or monthly, if you publish quarterly or monthly.
• Record count:  To get a record count, you can log into the journal as "Editor", click to "Back Issues", then tally up the number of articles in the back issues.  Remember, if you are excluding some content, for example, excluding full issue PDFs while including article PDFS, then take that into account.

Linking Questions:

• Preference for linking to the metadata record, full text, or both:  Choose which option you prefer.  Download and usage statistics are available in Florida OJ, regardless of whether the person goes directly to the PDF.
• Does an element link back to the metadata record?:
  • Yes.
  • In MARC, this is 856u.
  • In Dublin Core, this is dc:identifier .
• Does an element link back to the full text/image/document?
  • Yes.
  • This field is not available in MARC.
  • In Dublin Core, this is dc:relation .
• If direct links are not provided, can they be constructed?:  No.

Data Questions:

• What element represents a persistent and unique identifier?:
  • In MARCXML or in Dublin Core, this is in the header provided as part of the OAI-PMH standard.  It is in record/header/identifier .
• Which data element represents publication date?:
  • In MARC, this is 260c.
  • In Dublic Core, this is dc:date .
• What data element represents "date of entry into the electronic collection"?:
  • In MARCXML or in Dublin Core, this is in the header provided as part of the OAI-PMH standard.  It is in record/header/datestamp .
• Are there data elements that should be searched as subjects, but are not part of the <Subject> element?:
  • No.
  • The keywords specified in the Florida OJ record are not included in the MARC record. 
  • In Dublin Core, all are included in dc:subject .
• If your data contains separate multiple creators, please note which are to be listed as creators and which as contributors:
  • As of Spring 2019, based on an analysis of records in Florida OJ, all journals are using the author field to include authors only and not other types of contributors.  Therefor, all should be listed as creators.
• If you have a dtd, please attach:  There is no need to attach a dtd.  The metadata coming out of Florida OJ through OAI-PMH harvests is in standard formats, and not a local format.

Collection Setup Questions:

• What element contains the "Collection Name" or equivalent set name?
  • OJS is unique in this regard.  Each journal has a separate OAI-PMH repository.  So, the URL of the OAI-PMH repository separates one journal from another.  Sets in Florida OJ refer to sections of the journal.  For example, if you journal has a section for "Front Matter", "Back Matter", and "Articles", then those are your sets.  You probably do not want those to show up as Collections or facets in EDS.  Instead, you might want to do something like harvest only "Articles", but not "Front Matter", "Back Matter", nor "Full Issue PDFs".
  • FLVC/FALSC recommends you browse the journal, look at journal sections, then if you want to exclude specific sections, describe that.  Most journals will want to harvest all sets.  Please feel free to contact help@flvc.org if this is unclear.
• Please submit a tab delimited look up table that maps Collection Names to what should be displayed in the Collection facet, Collection limiters, and on the Full Record display:  Not applicable.  Each journal should appear as a facet (ie. each OAI-PMH feed coming out of Florida OJ should appear as a facet).  Sections within journals (ie. sets in OAI-PMH) should not normally be displayed as facets, however,  you may wish to include some content from harvest at all; for example, you may wish to exclude "Front Matter" from harvesting at all.

Setup for Institutional Repository/Digital Archives Publication Type Lookup Table: 

• Many digital libraries contain mixed types of content - ie. old photos, cultural artifacts, academic publications.  Journals in Florida OJ overwhelmingly contain Academic Journal content.  Nevertheless, Conference Proceedings, Periodicals, and Trade Publications are also represented.  It is recommended that you look at the table provided by EDS, and write an explanation for each journal describing which content that journal contains.

The Sunshine State Digital Network (SSDN) is the Digital Public Library of America's (DPLA) hub for Florida.  SSDN harvests records from digital libraries around the state and sends them on for display in the DPLA.  You can see Florida's contributions to the DPLA here https://dp.la/search?partner%5B%5D=Sunshine+State+Digital+Network .

In order to apply for participation through the SSDN, fill out the Content Contributor Application here https://sunshinestatedigitalnetwork.wordpress.com/how-to-participate/ for each digital library platform and for each journal in Florida OJ (each journal in Florida OJ has its own OAI-PMH feed).

Metadata guidance for SSDN participation:

• Articles vs full issues:
  • Regarding Florida OJ specifically, please refer to the DPLA's Collection Development Guidelines at https://pro.dp.la/hubs/collection-development-guidelines .  As of spring 2019, guidelines for Serials state that Serials are generally excluded when metadata is at the issue level, but generally included when metadata is at the journal level.  Guidelines for Scholarly Materials state that, "DPLA is generally interested in adding to the collection materials that have been published or contain textual descriptions of research such as article pre- and post-prints."   Florida OJ provides metadata at the journal article level (assuming you have uploaded a separate file for each article, rather than a single issue-level file).  This would seem to meet the guidelines.  Journals submitting to SSDN are encouraged to write to help@flvc.org and let FLVC/FALSC know what feedback was received and whether journal content is represented in the DPLA.
• Copyright statement:
  • SSDN requires a copyright statement.  For instructions on adding this to Florida OJ, please refer to http://falsc.libguides.com/coprightstatementsdigitalandoer/RightsStatementsInFloridaOJ .  Florida OJ supports adding a standard copyright statement across articles, and support Creative Commons licenses.

Guidance for the SSDN's Content Contributor Application:

• Question:  Which content management system does your organization use?
  • Answer:  Open Journal Systems
• Question:  In which format is your metadata available?
  • Answer:  Dublin Core, MARC
• Question:  Do you have an OAI-PMH endpoint?
  • Answer:  Yes
• Question:  If yes to the above, what is your OAI-PMH endpoint's address?
  • Answer:  To find an OAI-PMH repository for your journal, add /oai? to the end of the URL.  For example, for the journal https://journals.flvc.org/FAU_UndergraduateLawJournal , the OAI-PMH repository is https://journals.flvc.org/FAU_UndergraduateLawJournal/oai?
• Question:  What is the public URL to your collections?
  • Answer:  Provide a link to your journal's homepage on Florida OJ.
• Question:  How many records do you anticipate sharing?
  • Answer:  Count how many published and publicly available articles you have, and provide that number.

A demo statewide digital collections search is available at http://testcat.flvc.org .  As of Spring 2019, content from The Democratic Communiqué ( https://journals.flvc.org/demcom ) is posted to http://testcat.flvc.org  .

Demo/test Statewide Digital Collections search:

• At this time, feedback on how records display is encouraged.  Please send feedback to help@flvc.org with "Florida OJ" in the subject line.
• Things to notice / open issues:
  • When a full issue is uploaded as an article, the article title can be repetitive (ie. the journal's name appears as the title in several records).  This is addressed in the best practices section of this LibGuide.
  • Subjects in the demo/test search display as "Array".  What should be showing there is a list of the keywords for the article in Florida OJ.