Component id	P2KG-Pipeline
Type	Software
Name	P2KG - Pattern to Knowledge Graph JAMS Pipeline
Description	This code takes the patterns generated by the FoNN tool in the form of pickle file and then creates knowledge graph of all the patterns found
Work package	WP3
Project	polifonia-project
Demo	https://projects.dharc.unibo.it/melody/fonn/statistics_on_the_session_annotated_subset_and_meertens_tune_collections_mtcann_pattern_kg
Release date	01/12/2022
Release number	v0.1.0.1-dev
Licence	GNU_GPL_v3
Contributors	Danny Diamond Abdul Shahid James McDermott
Related components	Reuses: SPARQL Anything Command Line (CLITool) Extends: The Smashub pipeline exemplified in Choco https://github.com/smashub/choco
Bibliography	Main publication: Forthcoming paper in SWODCH 2023 Publication: Polifonia Deliverable 3.4
Credits	This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement N. 101004746

error URL cannot be resolved. (error)

View this file on Github Download as Schema.org

P2KG - Patterns to Knowledge Graph JAMS Pipeline

Targeting the goals of Polifonia WP3 package, P2KG JAMS Pipeline creates the knowledge graph of the patterns generate by FoNN. The details of directory and files are given below:
Directories
- config
  - config.yml => this file contains configurations related to pickle file location and JAMS annotations such as music_pattern_directory and corpus_name etc. You need to update these settings before executing the pipeline
  - jams2rdf_config.yml => this file contains JAMS to RDF process settings, for example, rdf_directory instruct code where to create the rdf files.
- music_patterns_pickle
  - thesession_27_04_23_kg_data.pkl => this file contains pattern information in each tune. This file must follow the following structure
    - id_number => contains the tune id
    - tune_title => contains the tune title information
    - tune_family => tune family information
    - feature => such as “diatonic pitch class”
    - level => information such as “accent” or “note” level
    - n_vals => n-gram information such as “(4, 5, 6)”
    - duration_beats => the tune duration length, e.g. 34
    - locations => pattern information should follow this structure, Dictionay of patterns in tuple and then its locations in a the list, e.g., “{(3, 4, 3, 2): [0, 8], (4, 3, 2, 3): [1, 9]}”
    - feature_sequence_data=> feature sequent data, list of pitch class values, [3, 4, 3, …. ]
  - thesession_metadata.csv => This file contains metadata of each tune, it should follow the following structure
    - X => contains the tune id
    - title=> contains the tune title information
    - Z => transcriber of the tune
    - R => tunetype information
    - M => timesignature information
    - K => key of the tune
    - score => content of the tune in the form of ABC notation
    - Formatted_title => clean tune title’s information
- schemas
  - pattern_fonn.json => this is a JAMS schema file, it is required for creating proper JAMS file. For each tune (in the pickle file) a corresponding JAMS file will be generated.
- sparql_anything
  - jams_ontology_pattern.sparql => this is the query the SPARQL Anything Engine require to create an RDF file for a given JAMS file.
  - sa.jar => this is SPARQL Anything engine, you can download the latest release from SPARQL-Anything GitHub link.
- tests => this folder contain test cases (TODO- to be developed)
- JAMS=> .jams file will be created in this folder for a corpus, for example, all The Session corpus files will go inside “thesession” folder
- RDF => All RDF files, .ttl files will be created inside this folder. for example, all The Session corpus files will go inside “thesession” folder

NOTE: Apart from these folders, you will find the following important .py files

pattern2kg_pipeline.py => this is the starting file, you can start process by executing this file. It will first create all the jams files and then it will create rdf files

pattern2kg_pipeline_parallel.py => this file can be used if you want to execute the whole process in parallel (parallelized version of the above file). However, most of the time you will not need to run this file.

pickle2jams.py=> this file is responsible for creating .jams files in the JAMS folder

jams2rdf.py => this file is responsible for creating .ttl files in the RDF folder

P2KG JAMS Pipeline:

P2KG - General Steps
- 1.1. Input: Patterns generated using FoNN in the form of pickle file.
- 1.2. Process: [JAMS Annotation] (https://jams.readthedocs.io/en/stable/) - JAMS files are created using custom pattern schema, you can find in the schema folder.
- 1.3. Process: [SPARQL Anything Engine] (https://sparql-anything.cc/) - It takes JAMS file and creates the KG.
- 1.4. Output: Knowledge Graph of pattern is created, a demo of created KG can be accessed on [Polifonia server] (https://polifonia.disi.unibo.it/fonn/sparql).

Outputs

After processing one tune to a JAMS file and then to RDF, we have output like the following. We have one MusicalComposition stanza:

<http://w3id.org/polifonia/resource/MusicalComposition/29318d2d8c8d2faebfb1e1560fb208c57681d91a>
        rdf:type             mc:MusicalComposition ;
        rdfs:label           "Drowsy Maggie" ;
        jams:beatsDuration   "34.0"^^xsd:float ;
        jams:key             <http://w3id.org/polifonia/resource/key/> ;
        jams:timeSignature   <http://w3id.org/polifonia/resource/timesig/> ;
        jams:tuneContent     "" ;
        jams:tuneFamily      <http://w3id.org/polifonia/resource/tunefamily/Drowsy_Maggie> ;
        jams:tuneId          "27" ;
        jams:tuneType        <http://w3id.org/polifonia/resource/tunetype/> ;
        mc:title             "Drowsy Maggie" ;
        prov:wasDerivedFrom  <http://w3id.org/polifonia/resource/JAMSFile/29318d2d8c8d2faebfb1e1560fb208c57681d91a> .

And one JAMSFile:

<http://w3id.org/polifonia/resource/JAMSFile/29318d2d8c8d2faebfb1e1560fb208c57681d91a>
        rdf:type             jams:JAMSFile ;
        jams:beatsDuration   "34.0"^^xsd:double ;
        jams:jamsVersion     "0.3.4" ;
        jams:release         "n-grams patterns-kg 1.0" ;
        prov:wasDerivedFrom  "The Session" ;
        prov:wasMemberOf     <https://github.com/ashahidkhattak/thesession> .

And then many JAMSObservations:

<http://w3id.org/polifonia/resource/JAMSObservation/9d82e4a5005124ece8d7e9472f5e71dedbe26ea6>
        rdf:type                   jams:ScorePatternOccurrence , jams:JAMSScoreObservation , jams:JAMSObservation ;
        jams:hasPatternComplexity  "1.0"^^xsd:float ;
        jams:hasPatternLocation    "3.0"^^xsd:float ;
        jams:hasPatternType        "feature='diatonic scale degree', level='accent', n_vals=4" ;
        jams:ofPattern             <http://w3id.org/polifonia/resource/pattern/5_2_1_6> .

P2KG - Requirements

To ensure P2KG runs correctly, please install the following libraries:

pip install -r requirements.txt

Attribution

If you use the code in this repository, please cite this software as follow:

@software{patterns2Kg_jams_pipeline_2022,
	address = {Galway, Ireland},
	title = {{P2KG} - {P}attern-2-{KG} {Knowledge Graph}},
	shorttitle = {{P2KG}},
	url = {https://github.com/polifonia-project/P2KG},
	publisher = {National University of Ireland, Galway},
	author = {Shahid Abdul, Diamond Danny, McDermott, James and Pushkar Jajoria},
	year = {2022},
}

License

This work is licensed under CC BY 4.0, https://creativecommons.org/licenses/by/4.0/