MbEventS 0.9.42

Beta

This project is in beta phase. APIs may change.

MbEventS

MbEventS (MusicBrainz Event Seeder) is a c# library utilizing the command line version of yambs to create seeds for the MB data base. There is a public API with some methods for various situations.

Setup

On the target system golang and yambs needs to be installed and part of your PATH. Follow the instructions here to get set up.

The instructions may vary depending on the operating system.

Linux (Ubuntu)

• Add golang to you PATH (add this line to ~/.profile) PATH="$PATH:/snap/go/current/bin:$HOME/go/bin"

  There may be many bash/profile files. If there is a empty .bash_profile file, just delete it that the .profile file is used.

Input Formats

The supportet input formats are descibed below. They can be divided into two groups. Text input and a list of web scrapers. No matter which input method used, every edit needs to follow the styleguide. Please make sure to check the seed befor submitting.

Text Input

There are several methods in the class TextInputController.cs Basically they all take a string as input and a path as output directory. Some have additional parameters.

Band Wave

Adds a band wave (multiple artists announced together) to an existing Event. The name of the file has to contain the string "BandWave". In example "BandWave_XX.txt" You can also use "Complete" instead of the "BandWave" string. The input should look like this:

edit_note=Band Wave XX, <Date of information>, <Link to source of the information>
mbid=b4a04573-788f-4938-b6dc-f506b9da1dfc
begin_date=2024-06-21
end_date=2024-06-23
Iron Maiden -- Iron Maiden Anniversary Show
Megadeth
... list with artist names of any length

A line with an artist name can contain a "credited as" note prefixed by " -- " mbid: MusicBrainz Id of the event where the artists should be added. Can be fount in the URL of the event. If not specified, a new event will be created. begin_date and end_date: optional, Format: yyyy-MM-dd

Day sub events

Adds day subevents to an existing (main) event. Can be used before adding the Running Order to get more control over the created day sub events. The name of the File has to contain the string "DaysSubEvents". In example "DaysSubEvents.csv".

edit_note="Added days"
rel0_type=818
rel0_target=b4a04573-788f-4938-b6dc-f506b9da1dfc
rel0_backward=true
rel1_type=794
rel1_target=3acd3d34-e8ea-4b9d-be0f-bd1e128aa8ba
"Event Name Year, Day X",BeginDate,EndDate,DoorsTime

In example

...
"Wacken Open Air 2023, Day 1",2023-07-31,2023-07-31,17:00
"Wacken Open Air 2023, Day 2",2023-08-01,2023-08-01,11:00
"Wacken Open Air 2023, Day 3",2023-08-02,2023-08-02,11:00
"Wacken Open Air 2023, Day 4",2023-08-03,2023-08-03,11:00
"Wacken Open Air 2023, Day 5",2023-08-04,2023-08-04,11:00
"Wacken Open Air 2023, Day 6",2023-08-05,2023-08-05,11:00

rel0_target: MusicBrainz Id of the event where the day sub events should be added. rel1_target: MusicBrainz Id of the place where the event was held.

Running Order (Schedule)

Adds stage sub events to the already existing day sub events. Performing artists are included. Also the stage place can be included. The name of the File has to contain the string "Schedule". For example, "Schedule.txt".

edit_note=https://www.wacken.com/en/program/running-order-music/#/

day_event_mbid=b1d7bf12-85ee-4f09-bb47-24183f704189
stage_place_mbid=291c9e90-def1-4f9e-bb07-1a8bc8b678f1
17:00-18:00 Acoustic Steel
19:00-20:00 Rock Stories
21:00-21:30 Alien Rockin Explosion
21:31-23:00 Maschine's Late Night Show

day_event_mbid=2011c3bc-d278-4cc6-88b0-f0b6cbda5a30
stage_place_mbid=291c9e90-def1-4f9e-bb07-1a8bc8b678f1
11:00-12:00 Metal Yoga
13:00-14:00 Kumpels in Kutten
15:00-16:00 Mit Ohne Strom
17:00-18:00 Uwe Bahn -- Uwe Bahn We Will Rock You
19:00-20:00 TBA
21:00-21:30 Alien Rockin Explosion
21:31-23:00 Maschine's Late Night Show
stage_place_mbid=5c701aca-e199-4d95-ad77-16b4a5ed21da
12:00-12:45 Skew Siskin
14:00-15:00 Nervosa
15:45-16:45 Holy Moses
17:30-18:45 Battle Beast
19:45-21:00 Beyond The Black
22:00-00:00 Doro -- Doro 40th Anniversary Show

day_event_mbid=b25c920d-31cf-413f-9179-10e4c23256a4
stage_place_mbid=49a5676c-8b14-46e4-8680-a09ce43505b3
12:45 - 13:45 Delain
15:15 - 16:15 Jinjer
17:45 - 19:00 Killswitch Engage
20:45 - 22:15 Heaven Shall Burn
00:15 - 01:30 Dropkick Murphys
stage_place_mbid=fdec8e0f-55fe-4c90-9b93-19f444de0e21
11:30 - 12:30 Masterplan -- Masterplan 20th Anniversary Show
14:00 - 15:00 Ensiferum
16:30 - 17:30 Alestorm
19:15 - 20:30 Saltatio Mortis
22:15 - 22:44 TBA
22:45 - 00:00 Two Steps From Hell

If you need further explanation read the other formats above.

Tour events (concerts)

Common information (Obsolete, just used for BandWave)

This file is not direct input file. It can be used to map a series and place id to a festival. Some methods use it if availiable to improve lookup. The file has to be named "Common.txt". It should be placed on directory above (../) the input file.

series_mbid=551d8620-6a43-4d00-84e6-544515c9024e
place_mbid=d87f6bbb-8f02-45dd-abaf-4a3b291b3b5d

Web Scraper

This one is more complex but also more automated. If the scraper is written once it can be used until the festival page changes.

The input consists of two parts. A scraper input file containing information about multiple scraper and an implementation of IFestivalScraper for each Scraper specified in the ScraperInput.csv file. The binary (DLL) containing the IFestivalScraper implementations can be placed beside the ScraperInput.csv file or beside the executing assembly (MbEventS.Console).

In example:

0,Vainstream,ea3b7e53-0acb-4448-9f40-fc6d30d706d3
3,Wacken,97bbae22-34ad-44c1-99ba-bce104c73dd1

<Mode>,<ScraperName>,<SeriesMbId>

Mode

Festivals can be in different modes. They start with Lineup announcements, so called Band Waves. Then at some time the Running Order will be presented. Sometimes all Slots at once, sometimes in steps. Corrections can also happen. Even last minute before the gig. Running Order mode ends, when the Festival ends. Then the festival enters pause mode until the Linup announcements start for the next episode.

There are the following modes:

• Pause Mode: The festival is in this mode before the first season starts or after the last season has ended. The mode identifier, the number in the fist column of the file ScraperInput.csv, is set to 0.
• Lineup Announcement Mode: A festival is in this mode while more and more artists are confirmed. To prepare this mode, we have to create an Event (Festival) as part of its Series. A held at Place relationship is recommended. Set the mode identifier to 1.
• Linup Complete Mode: Festival says that the lineup is complete, but Running Order Mode has not started. Before setting this state you should manually check the correctness. Set the mode identifier to 2.
• Running Order Mode: Usually the running order will be published as a whole. It is clear when and where the artists will perform. Day, time and stage. To prepare this mode, we have to create an Event for each Day with performing Artists as part of the Main Event. If it is a one Day festival you can skip the Day sub-Events. Set the mode identifier to 3.

To maintain a festival we need to create a Series (Festival) and add it to the collection Maintained Festival Series for FestivalGuide App (Please contact me there to avoid to write scrapers twice). A held at Place relationship is recommended. This Place should be named "<FestivalName> Festival Grounds" in example "Wacken Open Air Festival Grounds". If it is planned to add Running Order data, it will be good to add a part (Place) relationship for each stage.

Scraper Name

ScraperName in the ScraperInput.csv file, the Name of the class implementing IFestivalScraper and the value of the Property ScraperName must match exactly.

Series MbId

MusicBrainz id of the corresponding series.

Create Scrapers

Some programming skills are required to create a scraper.

Setup

You need to create a C# dotnet library project. You can mostly copy this content to the .csproj file of the just created project. Make sure you read the comments in the exmaple below.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <RootNamespace>MbEventS.Scraper</RootNamespace>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.2" />
    <PackageReference Include="Selenium.WebDriver" Version="4.40.0" />
  </ItemGroup>

  
  <Choose>
    <When Condition="'$(Configuration)' == 'Debug'">
      <ItemGroup>
        <ProjectReference Include="..\MbEventS\MbEventS.csproj"/>
      </ItemGroup>
    </When>
    <Otherwise>
      <ItemGroup>
        <PackageReference Include="MbEventS" Version="0.9.37"/>
      </ItemGroup>
    </Otherwise>
  </Choose>

  <ItemGroup Condition="'$(Configuration)' == 'Debug'">
    <None Update="ScraperInput.csv" CopyToOutputDirectory="PreserveNewest" />
  </ItemGroup>

  
  <Target Name="PostBuildScript" AfterTargets="Build" Condition="'$(Configuration)' == 'Debug'">
    <Copy SourceFiles="$(TargetPath)" DestinationFolder="$(TargetDir)../../../../MbEventS.Ui/bin/Debug/$(TargetFramework)/"/>
    <Copy SourceFiles="$(TargetDir)$(TargetName).pdb" DestinationFolder="$(TargetDir)../../../../MbEventS.Ui/bin/Debug/$(TargetFramework)/"/>
  </Target>

</Project>

Then you need to install or clone MbEventS.Ui. If you clone it, make sure to place it next to the Scraper project you have just created. After that you should create a solution or workspace to open all the projects together in one IDE.

<Solution>
  <Folder Name="/Files/">
    <File Path="..\ScraperInput.csv" />
  </Folder>
  <Project Path="<Your Scraper Project>\<Your Scraper Project>.csproj" Type="Classic C#" />
  <Project Path="MbEventS.Ui\MbEventS.Ui.csproj" Type="Classic C#" />
  <Project Path="MbEventS\MbEventS.csproj" Type="Classic C#" />
</Solution>

You should now have the following file structure:

repos
  |- Solution.slnx
  |- MbEventS <- Optional
  |  |-MbEventS.csproj
  |- MbEventS.Ui <- Optional
  |  |-MbEventS.Ui.csproj
  |- <Your Scraper Project>
  |  |-<Your Scraper Project>.csproj

For now the ScraperInput.csv must be placed in

// Ubuntu: /home/<user name>/ScraperInput.csv
Environment.GetFolderPath(Environment.SpecialFolder.UserProfile)

There will also be a directory created, named "ScraperOuput" containing the seeds after the run.

Implementing a scraper

To implement a festival scraper, you need to create a class implementing the IFestivalScraper interface.

In the constructor you need to set the following properties:

• this.logger = logger;
• ScraperName = "Bloodstock";
• ArtistUrl = new Uri("<url to the linup page>");
• ScheduleUrl = new Uri("<url to the running order page, can be the same as ArtistUrl>");;

There are three methods to implement:

• void ScrapeDetails(ArtistInfo artistInfo)
• EventInfo ScrapeEvent()
• EventInfo ScrapeSchedule()

You should implement ScrapeEvent() for scraping band waves and ScrapeSchedule() for scraping the schedule.

You are totally free with the implementation of the methods mentioned above. But there are some pattern that have proven stability. The blow code can be taken as a recipie:

public class <Your scraper name>Scraper : IFestivalScraper
	{
		public <Your scraper name>Scraper(ILogger logger = null)
		{
			this.logger = logger;
			ScraperName = "<Your scraper name>";
			ArtistUrl = new Uri("<url to the linup page>");
			ScheduleUrl = new Uri("url to the running order page, can be the same as ArtistUrl");
        }

		public override EventInfo ScrapeEvent()
		{
			driver.Navigate().GoToUrl(ArtistUrl);

			var container = driver.FindElement(By.CssSelector("<select a continer element which wraps all artists>"));
			var artists = container.FindElements(By.CssSelector("<find a common element which selects each single artist"));
			var result = new EventInfo();

			foreach (var artist in artists)
			{
				var artistNameTag = artist.FindElement(By.XPath("<path to the artist name>"));
				var artistName = artistNameTag.GetAttribute("innerText");

				var linkTag = artist.FindElement(By.XPath("<path to the artist detail page url if available>"));
				var link = linkTag.GetAttribute("href");
                                
				var artistInfo = new ArtistInfo
				{
					Name = artistName,
					DetailPageUrl = link
				};

                // If the social media links are on the same page, you can call ScraperDetailsHref(container, artistInfo); here 
                // and leave void ScrapeDetails(ArtistInfo artistInfo) blank
                
				result.Artists.Add(artistInfo);
			}

			return result;
		}

		public override void ScrapeDetails(ArtistInfo artistInfo)
		{
			secondDriver.Navigate().GoToUrl(artistInfo.DetailPageUrl);
            
			var container = secondDriver.FindElements(By.CssSelector("<select a container containing all social media links for the artist>"));

			ScraperDetailsHref(container, artistInfo);
		}

		public override EventInfo ScrapeSchedule()
		{
			driver.Navigate().GoToUrl(ScheduleUrl);

			// Same as ScrapeEvent() but much more complicated. Depending on the page and event structure, you need different workflows.
            // The Idea is to fill the List<DaySubEventInfo> with DaySubEventInfo objects.
            // List<DaySubEventInfo>
            //  |- Date
            //  |- Doors <- Optional
            //  |- List<ArtistInfo> <- If single stage the artists can be stored here.
            //  |- List<StageSubEventInfos>
            //    |- Date
            //    |- Doors <- Optional, not the time of the first performing artist
            //    |- StageName
            //    |- List<ArtistInfo> <- If multi stage the artists must be stored here.
            //      |- Name
            //      |- DetailPageUrl
            //      |- Time
            //      |- StageType
            
            // result.DaySubEventInfos have to be ordered bei date ascending
			return result;
		}
	}

Output

The seeds will be written as html sites in the directory "seeds" in your downloads dir.

Developer Infos

Useful Ids

EventType_Festival EventType = 2

// Indicates that the event is part of a series.
LinkType_PartOf_Event_Series LinkType = 802

// This attribute indicates the number of an entity in a series.
LinkAttributeType_Number LinkAttributeType = 788 // number

// This indicates that an event is made up of multiple parts (e.g. a festival
// happening on multiple venues over the course of a few days).
LinkType_Parts_Event_Event LinkType = 818

// Links an event to the place where it was held.
LinkType_HeldAt_Event_Place LinkType = 794

// Links an event to (one of) its main performer(s).
LinkType_MainPerformer_Artist_Event LinkType = 798

// Links an event to (one of) its support act(s) (also known as opening acts or
// warm-up acts).
LinkType_SupportAct_Artist_Event LinkType = 799

// Local time a band's performance is scheduled to start, formatted HH:MM.
LinkAttributeType_Time LinkAttributeType = 830

Festivals

Running Order only as Image/PDF

There are some festivals where the running order is just available as an image or PDF. In this cases we my utilize an LLM which understands images like ChatGPT4.o. The image of the running order and a prompt like this may work:

I need the running order of the festival from the image in this format, in plain text. All days and stages in one block of text. 

the day_event_mbid of the festival are
```
ebdeb7dc-33f2-4add-a587-e16985a7b9ec
82c575b0-3a90-4d81-91c3-f377d752af09
f75f78fd-bae7-4e2b-993b-68e16f42fa03
```
stage_place_mbid for the stages of rock am ring are
```
EMP Mad Max Mainstage -> 5fb731c9-c0d0-4f92-b19f-0973de7526e5
Medusa Stage -> bf06af38-8376-451f-8db8-67ccbb867fc3
Hardbowl Tentstage -> df6c2f25-db33-4071-8ce8-134c5ecd03d9
Backyard Stage -> c3a176c1-6507-4da1-886c-7e41443fcafc
```
Result format:
```
edit_note=https://www.wacken.com/en/program/running-order-music/#/
day_event_mbid=b1d7bf12-85ee-4f09-bb47-24183f704189
stage_place_mbid=291c9e90-def1-4f9e-bb07-1a8bc8b678f1
17:00-18:00 Acoustic Steel
19:00-20:00 Rock Stories
21:00-21:30 Alien Rockin Explosion
21:31-23:00 Maschine's Late Night Show
day_event_mbid=2011c3bc-d278-4cc6-88b0-f0b6cbda5a30
stage_place_mbid=291c9e90-def1-4f9e-bb07-1a8bc8b678f1
11:00-12:00 Metal Yoga
13:00-14:00 Kumpels in Kutten
15:00-16:00 Mit Ohne Strom
17:00-18:00 Uwe Bahn -- Uwe Bahn We Will Rock You
19:00-20:00 TBA
21:00-21:30 Alien Rockin Explosion
21:31-23:00 Maschine's Late Night Show
stage_place_mbid=5c701aca-e199-4d95-ad77-16b4a5ed21da
12:00-12:45 Skew Siskin
14:00-15:00 Nervosa
15:45-16:45 Holy Moses
17:30-18:45 Battle Beast
19:45-21:00 Beyond The Black
22:00-00:00 Doro -- Doro 40th Anniversary Show
day_event_mbid=b25c920d-31cf-413f-9179-10e4c23256a4
stage_place_mbid=49a5676c-8b14-46e4-8680-a09ce43505b3
12:45-13:45 Delain
15:15-16:15 Jinjer
17:45-19:00 Killswitch Engage
20:45-22:15 Heaven Shall Burn
00:15-01:30 Dropkick Murphys
stage_place_mbid=fdec8e0f-55fe-4c90-9b93-19f444de0e21
11:30-12:30 Masterplan -- Masterplan 20th Anniversary Show
14:00-15:00 Ensiferum
16:30-17:30 Alestorm
19:15-20:30 Saltatio Mortis
22:15-22:44 TBA
22:45-00:00 Two Steps From Hell
```

Here a not complete list of festivals with image or PDF running order:

• FullForce
• Hexentanz
• High Flames
• Rock am Härtsfeldsee
• RaR
• RiP
• Rock das Ding
• Taubertal
• Leandas del Rock
• Vainstream