EPG discussion on Enigma2 STB's

[iBoston]

I've been doing a lot of work on my EPG program, and i wanted to share a discovery i made today.

I build the XML formatted file myself. The XML is formatted in XMLTV format for EPG data.

<programme start="20230205130000 -0500" stop="20230205140000 -0500" channel="CoziTV"> <title lang="en">Little House on the Prairie</title> <desc lang="en">A Harvest of Friends : [TVPG:60 Min]</desc> </programme>

Well, during programming, i came upon a quark i didn't expect. I was building my XML file grouped by DAY, rather then CHANNEL for a particular source. So, the outputted XML file would list all the channels and programs for one day, and then list all the channels and programs for the next day and so on. Utilizing the EPGimport plugin on the Enigma2 STB, i realized that it was only showing EPG data for the last date in the xml file, missing all earlier dates. Come to figure out, that something doesn't like channel data separated. If you move on to another channel, and then back to an existing channel, the earlier channel data is tossed. So, it requires the XML data to be grouped by channel ID. -- This might be a limitation of the EPGimport or a mandatory XMLTV format specification. Either way, i needed to create a fix.

I wrote a function that runs through the EPG array (before writing the xml file) and re-group the ARRAY by channel ID rather than date. A sort function so to speak. I then rebuilt the XML file with my GroupByID function called first and the EPGimport plugin ran flawlessly showing all EPG data inclusive of all dates.

 

[iBoston]

For years now, i have had EPG information on all my channels. I just recently did a major code modification to my program to add additional sources (where i fetch my EPG data) which allowed me to dump what was once my primary source that was limited to 6 hours of events. I now have the ability to fetch 14 days of events for all my channels, although, i really only reserve that for my favorites, and only fetch 2 days of events for all others.

I cannot begin to tell you how nice EPG data within the STB is addictive. Being i have two tuners, i can record two things at a time, or watch one thing and record another. With EPG data on my favorite channels for 14 days, its so easy for me to peruse through and say, oh that sound interesting, and click the 'Record Event', and then forget about it. I'm in love and don't ever want to go back to the days without EPG.

The process to make this work comes in 3 forms.

1. A source to grab the EPG data. (Such as schedule direct - per year subscription) or if your a coder, you can fetch your own with ingenuity.

2. A means to convert the EPG data to XMLTV formatted XML. (EPG information in XML format). This can be accomplished with (XMLTV - an open source free Linux or Windows program) or again, if your a coder, you can create the XMLTV formatted XML files yourself.

3. The Free Enigma2 EPGimport plugin that installs on your STB that imports the XML files and imports them as EPG data to be seen on your Satellite STB/TV.

If you have desire and some coding/scripting or just plain technical savvy, you can accomplish it. FTA4PA utilizes linux, and i utilize windows.

I sent an email to schedule direct inquiring about the feasibility of sharing those XMLTV formatted files. Either by giving permission to share, or maybe Schedule Direct providing those files directly. I would envision two XML files. One for Cband one for KU band. - We shall see if they respond with future potential.

 

[iBoston]

Here is a repost of FTA2PA private message to me. He gave me permission to post this. This is an explaination of how he implemented it himself utilizing schedule direct and the resources i mentioned above : I will add that, even though we can help, it can be task intensive and we have lives too. If you don't think you can tackle something like this, we cannot do it for you. But, if you have questions, feel free to post them, and we would be happy to post suggestions or insights to keep you on track :

There are several steps in getting it to work properly. It took me quite a bit of web research and a bit of trial and error until I figured it out.
Schedules Direct guide data is provided in XML and JSON. I have never tried JSON so I can't speak to that. I use XML as it works great with the EPGImporter plugin. Once you have a membership on Schedules Direct you can select up to four zip codes to get your guide data from. You then select any channels within those that you want guide data for. It takes a bit of research in deciding which channels you want from the list. Basically, they are presented as only the station id's so you must do research on the web to find out which channel each carries. It is a bit time consuming but once it is done you won't need to do it again unless the station changes what it carries or rearranges the channel order. I will use zip code 70826 for illustrating the process. It has the LPB guide data.
On the Schedules Direct website for that zip code you are presented with a list like this:
WBRZDT 2-2WBRZDT2 2-3WBRZDT3 3-1KATCDT
3-2KATCDT2 3-3KATCDT3 3-4KATCDT4 3-5KATCDT5
9-1WAFBDT 9-2WAFBDT2 9-3WAFBDT3 9-4WAFBDT4
9-5WAFBDT5 9-6WAFBDT6 10-1KLFYDT 10-2KLFYDT2
10-3KLFYDT3 10-4KLFYDT4 14-1KPBNLD 20-1KZUPCD
21-1WBRLCD 27-1WLPBDT 27-2WLPBDT2 27-3WLPBDT3
30-1WLFTCD 30-2WLFTCD2 30-3WLFTCD3 30-4WLFTCD4
33-1WVLADT 33-2WVLADT2 33-3WVLADT3 36-1KBTRCD
36-2KBTRCD2 36-3KBTRCD3 39-1WBXHCD 39-2WBXHCD2
39-3WBXHCD3 39-4WBXHCD4 44-1WGMBDT 44-2WGMBDT2
44-3WGMBDT3 48-1W48DWD 50-1KLWBDT 50-2KLWBDT2
50-3KLWBDT3 50-4KLWBDT4 50-5KLWBDT5
From this list 27-1WLPBDT, 27-2WLPBDT2 and 27-3WLPBDT3 have the data for the three LPB channels. Once you have selected all your zip codes and chosen/saved all your desired stations you are pretty much done on the Schedules Direct website and will move on to acquiring the actual guide data on your computer.
First you will need to generate a .dat configuration file for each of the zip codes you selected. You do this by running the following Linux command for each one:
tv_grab_na_dd --config lineup1.dat --configure

You don't need to use lineup1 for the file name. I name my .dat files for the channels I am getting so it is easier to keep track of what comes from where. My command is actually:

tv_grab_na_dd --config lineup_lpb.dat --configure

You will then select the corresponding lineup you want to associate with the .dat file and answer a few configuration questions. I believe one of these is about storing your password. It is not recommended to store it so I didn't. After answering everything it will generate the .dat file called lineup1.dat (or whatever name you gave it) containing the following:
username: your Schedules Direct user name would be here
timeoffset: -0400
lineup: PC:70826
auto-config: add
channel: 27-1 WLPBDT
channel: 27-2 WLPBDT2
channel: 27-3 WLPBDT3
You then use the following command in Linux to get the actual guide data from each .dat file you generated. I have this process automated with a BASH script I wrote that runs weekly:
tv_grab_na_dd --config lineup_lpb.dat --days 8 --dropbadchar --output schedule_lpb.xml < pw.txt
In that command line you have:
tv_grab_na_dd <-- Pretty self explanatory - the actual Linux command for grabbing the data
--config lineup_lpb.dat <-- The channel lineup to be downloaded.
--days 8 <-- Number of days of guide data to acquire. I do eight days so I can schedule recordings
--dropbadchar <-- Rids the XML file of some special characters than can (and usually will) cause problems when the guide data is read into your receiver.
--output schedule_lpb.xml <-- Your generated XML guide data file for use with EPGImporter.
< pw.txt <-- A text file (stored locally) containing only your Schedules Direct password. The contents are fed to the command so you don't have to enter it manually. Needed when automating getting your guide data.

 

[FTA4PA]

Here is a repost of FTA2PA private message to me. He gave me permission to post this. This is an explaination of how he implemented it himself utilizing schedule direct and the resources i mentioned above : I will add that, even though we can help, it can be task intensive and we have lives too. If you don't think you can tackle something like this, we cannot do it for you. But, if you have questions, feel free to post them, and we would be happy to post suggestions or insights to keep you on track :

There are several steps in getting it to work properly. It took me quite a bit of web research and a bit of trial and error until I figured it out.
Schedules Direct guide data is provided in XML and JSON. I have never tried JSON so I can't speak to that. I use XML as it works great with the EPGImporter plugin. Once you have a membership on Schedules Direct you can select up to four zip codes to get your guide data from. You then select any channels within those that you want guide data for. It takes a bit of research in deciding which channels you want from the list. Basically, they are presented as only the station id's so you must do research on the web to find out which channel each carries. It is a bit time consuming but once it is done you won't need to do it again unless the station changes what it carries or rearranges the channel order. I will use zip code 70826 for illustrating the process. It has the LPB guide data.
On the Schedules Direct website for that zip code you are presented with a list like this:
WBRZDT 2-2WBRZDT2 2-3WBRZDT3 3-1KATCDT
3-2KATCDT2 3-3KATCDT3 3-4KATCDT4 3-5KATCDT5
9-1WAFBDT 9-2WAFBDT2 9-3WAFBDT3 9-4WAFBDT4
9-5WAFBDT5 9-6WAFBDT6 10-1KLFYDT 10-2KLFYDT2
10-3KLFYDT3 10-4KLFYDT4 14-1KPBNLD 20-1KZUPCD
21-1WBRLCD 27-1WLPBDT 27-2WLPBDT2 27-3WLPBDT3
30-1WLFTCD 30-2WLFTCD2 30-3WLFTCD3 30-4WLFTCD4
33-1WVLADT 33-2WVLADT2 33-3WVLADT3 36-1KBTRCD
36-2KBTRCD2 36-3KBTRCD3 39-1WBXHCD 39-2WBXHCD2
39-3WBXHCD3 39-4WBXHCD4 44-1WGMBDT 44-2WGMBDT2
44-3WGMBDT3 48-1W48DWD 50-1KLWBDT 50-2KLWBDT2
50-3KLWBDT3 50-4KLWBDT4 50-5KLWBDT5
From this list 27-1WLPBDT, 27-2WLPBDT2 and 27-3WLPBDT3 have the data for the three LPB channels. Once you have selected all your zip codes and chosen/saved all your desired stations you are pretty much done on the Schedules Direct website and will move on to acquiring the actual guide data on your computer.
First you will need to generate a .dat configuration file for each of the zip codes you selected. You do this by running the following Linux command for each one:
tv_grab_na_dd --config lineup1.dat --configure

You don't need to use lineup1 for the file name. I name my .dat files for the channels I am getting so it is easier to keep track of what comes from where. My command is actually:

tv_grab_na_dd --config lineup_lpb.dat --configure

You will then select the corresponding lineup you want to associate with the .dat file and answer a few configuration questions. I believe one of these is about storing your password. It is not recommended to store it so I didn't. After answering everything it will generate the .dat file called lineup1.dat (or whatever name you gave it) containing the following:
username: your Schedules Direct user name would be here
timeoffset: -0400
lineup: PC:70826
auto-config: add
channel: 27-1 WLPBDT
channel: 27-2 WLPBDT2
channel: 27-3 WLPBDT3
You then use the following command in Linux to get the actual guide data from each .dat file you generated. I have this process automated with a BASH script I wrote that runs weekly:
tv_grab_na_dd --config lineup_lpb.dat --days 8 --dropbadchar --output schedule_lpb.xml < pw.txt
In that command line you have:
tv_grab_na_dd <-- Pretty self explanatory - the actual Linux command for grabbing the data
--config lineup_lpb.dat <-- The channel lineup to be downloaded.
--days 8 <-- Number of days of guide data to acquire. I do eight days so I can schedule recordings
--dropbadchar <-- Rids the XML file of some special characters than can (and usually will) cause problems when the guide data is read into your receiver.
--output schedule_lpb.xml <-- Your generated XML guide data file for use with EPGImporter.
< pw.txt <-- A text file (stored locally) containing only your Schedules Direct password. The contents are fed to the command so you don't have to enter it manually. Needed when automating getting your guide data.

As I shared with iBoston, it took me a lot of researching to glean the various bits of info needed from different sources to get this working under Linux for my Edision, including some very crucial info he provided regarding xml file structure. We both agree that setting this up is not a simple endeavour but it is well worth it IMHO. In addition to the above you will also need to define custom.channels.xml and custom.sources.xml files using the service ids for your FTA channels in your receiver and the location where you store your downloaded guide data.

I started out by acquiring my guide data manually, once a week, typing the necessary commands into the terminal to download my data from Schedules Direct. I then ran the EPGImporter plugin on my Edision to get it into the receiver. If you are like me, you will probably get tired of forgetting to do this. You can automate the whole process by using a script and having it run as a cron job and setting the EPGImporter plugin to grab the data automatically. I use a NAS drive for storage and the scripts I wrote are run on a computer I keep on 24/7.

Along with program start and end times most of the Schedules Direct epg data includes episode titles and program descriptions. I am in my second year as a subscriber and can highly recommend the service they provide to hobbyists like us. If you have the skills and are willing to put some effort into using the data they supply it is well worth the time and cost to get a reliable, working guide on your FTA receiver.

 

[iBoston]

If your looking for EPG for Patient Channel (they update it every quarter year) although it rarely changes much... So, even if you update it once a year, it won't be off by much. :

Program Guides

Program guides are shipped quarterly and should be used with every patient to help promote your patient education resources.

www.thewellnessnetwork.net

I also included an XML file i created that breaks it down to something closer to what EPGimport would need.. This is a template my program uses to build the EPG. The txt file breaks it down to a weekly schedule. My program then utilizes the template xml file to build date/start/stop times to comply with XMLTV format.

 

[FTA4PA]

If your looking for EPG for Patient Channel (they update it every quarter year) although it rarely changes much... So, even if you update it once a year, it won't be off by much. :

Program Guides

Program guides are shipped quarterly and should be used with every patient to help promote your patient education resources.

www.thewellnessnetwork.net

I also included an XML file i created that breaks it down to something closer to what EPGimport would need.. This is a template my program uses to build the EPG. The txt file breaks it down to a weekly schedule. My program then utilizes the template xml file to build date/start/stop times to comply with XMLTV format.

Thanks!