EPG discussion on Enigma2 STB's

[iBoston]

I've been doing a lot of work on my EPG program, and i wanted to share a discovery i made today.

I build the XML formatted file myself. The XML is formatted in XMLTV format for EPG data.

<programme start="20230205130000 -0500" stop="20230205140000 -0500" channel="CoziTV"> <title lang="en">Little House on the Prairie</title> <desc lang="en">A Harvest of Friends : [TVPG:60 Min]</desc> </programme>

Well, during programming, i came upon a quark i didn't expect. I was building my XML file grouped by DAY, rather then CHANNEL for a particular source. So, the outputted XML file would list all the channels and programs for one day, and then list all the channels and programs for the next day and so on. Utilizing the EPGimport plugin on the Enigma2 STB, i realized that it was only showing EPG data for the last date in the xml file, missing all earlier dates. Come to figure out, that something doesn't like channel data separated. If you move on to another channel, and then back to an existing channel, the earlier channel data is tossed. So, it requires the XML data to be grouped by channel ID. -- This might be a limitation of the EPGimport or a mandatory XMLTV format specification. Either way, i needed to create a fix.

I wrote a function that runs through the EPG array (before writing the xml file) and re-group the ARRAY by channel ID rather than date. A sort function so to speak. I then rebuilt the XML file with my GroupByID function called first and the EPGimport plugin ran flawlessly showing all EPG data inclusive of all dates.

 

[iBoston]

For years now, i have had EPG information on all my channels. I just recently did a major code modification to my program to add additional sources (where i fetch my EPG data) which allowed me to dump what was once my primary source that was limited to 6 hours of events. I now have the ability to fetch 14 days of events for all my channels, although, i really only reserve that for my favorites, and only fetch 2 days of events for all others.

I cannot begin to tell you how nice EPG data within the STB is addictive. Being i have two tuners, i can record two things at a time, or watch one thing and record another. With EPG data on my favorite channels for 14 days, its so easy for me to peruse through and say, oh that sound interesting, and click the 'Record Event', and then forget about it. I'm in love and don't ever want to go back to the days without EPG.

The process to make this work comes in 3 forms.

1. A source to grab the EPG data. (Such as schedule direct - per year subscription) or if your a coder, you can fetch your own with ingenuity.

2. A means to convert the EPG data to XMLTV formatted XML. (EPG information in XML format). This can be accomplished with (XMLTV - an open source free Linux or Windows program) or again, if your a coder, you can create the XMLTV formatted XML files yourself.

3. The Free Enigma2 EPGimport plugin that installs on your STB that imports the XML files and imports them as EPG data to be seen on your Satellite STB/TV.

If you have desire and some coding/scripting or just plain technical savvy, you can accomplish it. FTA4PA utilizes linux, and i utilize windows.

I sent an email to schedule direct inquiring about the feasibility of sharing those XMLTV formatted files. Either by giving permission to share, or maybe Schedule Direct providing those files directly. I would envision two XML files. One for Cband one for KU band. - We shall see if they respond with future potential.

 

[iBoston]

Here is a repost of FTA2PA private message to me. He gave me permission to post this. This is an explaination of how he implemented it himself utilizing schedule direct and the resources i mentioned above : I will add that, even though we can help, it can be task intensive and we have lives too. If you don't think you can tackle something like this, we cannot do it for you. But, if you have questions, feel free to post them, and we would be happy to post suggestions or insights to keep you on track :

There are several steps in getting it to work properly. It took me quite a bit of web research and a bit of trial and error until I figured it out.
Schedules Direct guide data is provided in XML and JSON. I have never tried JSON so I can't speak to that. I use XML as it works great with the EPGImporter plugin. Once you have a membership on Schedules Direct you can select up to four zip codes to get your guide data from. You then select any channels within those that you want guide data for. It takes a bit of research in deciding which channels you want from the list. Basically, they are presented as only the station id's so you must do research on the web to find out which channel each carries. It is a bit time consuming but once it is done you won't need to do it again unless the station changes what it carries or rearranges the channel order. I will use zip code 70826 for illustrating the process. It has the LPB guide data.
On the Schedules Direct website for that zip code you are presented with a list like this:
WBRZDT 2-2WBRZDT2 2-3WBRZDT3 3-1KATCDT
3-2KATCDT2 3-3KATCDT3 3-4KATCDT4 3-5KATCDT5
9-1WAFBDT 9-2WAFBDT2 9-3WAFBDT3 9-4WAFBDT4
9-5WAFBDT5 9-6WAFBDT6 10-1KLFYDT 10-2KLFYDT2
10-3KLFYDT3 10-4KLFYDT4 14-1KPBNLD 20-1KZUPCD
21-1WBRLCD 27-1WLPBDT 27-2WLPBDT2 27-3WLPBDT3
30-1WLFTCD 30-2WLFTCD2 30-3WLFTCD3 30-4WLFTCD4
33-1WVLADT 33-2WVLADT2 33-3WVLADT3 36-1KBTRCD
36-2KBTRCD2 36-3KBTRCD3 39-1WBXHCD 39-2WBXHCD2
39-3WBXHCD3 39-4WBXHCD4 44-1WGMBDT 44-2WGMBDT2
44-3WGMBDT3 48-1W48DWD 50-1KLWBDT 50-2KLWBDT2
50-3KLWBDT3 50-4KLWBDT4 50-5KLWBDT5
From this list 27-1WLPBDT, 27-2WLPBDT2 and 27-3WLPBDT3 have the data for the three LPB channels. Once you have selected all your zip codes and chosen/saved all your desired stations you are pretty much done on the Schedules Direct website and will move on to acquiring the actual guide data on your computer.
First you will need to generate a .dat configuration file for each of the zip codes you selected. You do this by running the following Linux command for each one:
tv_grab_na_dd --config lineup1.dat --configure

You don't need to use lineup1 for the file name. I name my .dat files for the channels I am getting so it is easier to keep track of what comes from where. My command is actually:

tv_grab_na_dd --config lineup_lpb.dat --configure

You will then select the corresponding lineup you want to associate with the .dat file and answer a few configuration questions. I believe one of these is about storing your password. It is not recommended to store it so I didn't. After answering everything it will generate the .dat file called lineup1.dat (or whatever name you gave it) containing the following:
username: your Schedules Direct user name would be here
timeoffset: -0400
lineup: PC:70826
auto-config: add
channel: 27-1 WLPBDT
channel: 27-2 WLPBDT2
channel: 27-3 WLPBDT3
You then use the following command in Linux to get the actual guide data from each .dat file you generated. I have this process automated with a BASH script I wrote that runs weekly:
tv_grab_na_dd --config lineup_lpb.dat --days 8 --dropbadchar --output schedule_lpb.xml < pw.txt
In that command line you have:
tv_grab_na_dd <-- Pretty self explanatory - the actual Linux command for grabbing the data
--config lineup_lpb.dat <-- The channel lineup to be downloaded.
--days 8 <-- Number of days of guide data to acquire. I do eight days so I can schedule recordings
--dropbadchar <-- Rids the XML file of some special characters than can (and usually will) cause problems when the guide data is read into your receiver.
--output schedule_lpb.xml <-- Your generated XML guide data file for use with EPGImporter.
< pw.txt <-- A text file (stored locally) containing only your Schedules Direct password. The contents are fed to the command so you don't have to enter it manually. Needed when automating getting your guide data.

 

[FTA4PA]

Here is a repost of FTA2PA private message to me. He gave me permission to post this. This is an explaination of how he implemented it himself utilizing schedule direct and the resources i mentioned above : I will add that, even though we can help, it can be task intensive and we have lives too. If you don't think you can tackle something like this, we cannot do it for you. But, if you have questions, feel free to post them, and we would be happy to post suggestions or insights to keep you on track :

There are several steps in getting it to work properly. It took me quite a bit of web research and a bit of trial and error until I figured it out.
Schedules Direct guide data is provided in XML and JSON. I have never tried JSON so I can't speak to that. I use XML as it works great with the EPGImporter plugin. Once you have a membership on Schedules Direct you can select up to four zip codes to get your guide data from. You then select any channels within those that you want guide data for. It takes a bit of research in deciding which channels you want from the list. Basically, they are presented as only the station id's so you must do research on the web to find out which channel each carries. It is a bit time consuming but once it is done you won't need to do it again unless the station changes what it carries or rearranges the channel order. I will use zip code 70826 for illustrating the process. It has the LPB guide data.
On the Schedules Direct website for that zip code you are presented with a list like this:
WBRZDT 2-2WBRZDT2 2-3WBRZDT3 3-1KATCDT
3-2KATCDT2 3-3KATCDT3 3-4KATCDT4 3-5KATCDT5
9-1WAFBDT 9-2WAFBDT2 9-3WAFBDT3 9-4WAFBDT4
9-5WAFBDT5 9-6WAFBDT6 10-1KLFYDT 10-2KLFYDT2
10-3KLFYDT3 10-4KLFYDT4 14-1KPBNLD 20-1KZUPCD
21-1WBRLCD 27-1WLPBDT 27-2WLPBDT2 27-3WLPBDT3
30-1WLFTCD 30-2WLFTCD2 30-3WLFTCD3 30-4WLFTCD4
33-1WVLADT 33-2WVLADT2 33-3WVLADT3 36-1KBTRCD
36-2KBTRCD2 36-3KBTRCD3 39-1WBXHCD 39-2WBXHCD2
39-3WBXHCD3 39-4WBXHCD4 44-1WGMBDT 44-2WGMBDT2
44-3WGMBDT3 48-1W48DWD 50-1KLWBDT 50-2KLWBDT2
50-3KLWBDT3 50-4KLWBDT4 50-5KLWBDT5
From this list 27-1WLPBDT, 27-2WLPBDT2 and 27-3WLPBDT3 have the data for the three LPB channels. Once you have selected all your zip codes and chosen/saved all your desired stations you are pretty much done on the Schedules Direct website and will move on to acquiring the actual guide data on your computer.
First you will need to generate a .dat configuration file for each of the zip codes you selected. You do this by running the following Linux command for each one:
tv_grab_na_dd --config lineup1.dat --configure

You don't need to use lineup1 for the file name. I name my .dat files for the channels I am getting so it is easier to keep track of what comes from where. My command is actually:

tv_grab_na_dd --config lineup_lpb.dat --configure

You will then select the corresponding lineup you want to associate with the .dat file and answer a few configuration questions. I believe one of these is about storing your password. It is not recommended to store it so I didn't. After answering everything it will generate the .dat file called lineup1.dat (or whatever name you gave it) containing the following:
username: your Schedules Direct user name would be here
timeoffset: -0400
lineup: PC:70826
auto-config: add
channel: 27-1 WLPBDT
channel: 27-2 WLPBDT2
channel: 27-3 WLPBDT3
You then use the following command in Linux to get the actual guide data from each .dat file you generated. I have this process automated with a BASH script I wrote that runs weekly:
tv_grab_na_dd --config lineup_lpb.dat --days 8 --dropbadchar --output schedule_lpb.xml < pw.txt
In that command line you have:
tv_grab_na_dd <-- Pretty self explanatory - the actual Linux command for grabbing the data
--config lineup_lpb.dat <-- The channel lineup to be downloaded.
--days 8 <-- Number of days of guide data to acquire. I do eight days so I can schedule recordings
--dropbadchar <-- Rids the XML file of some special characters than can (and usually will) cause problems when the guide data is read into your receiver.
--output schedule_lpb.xml <-- Your generated XML guide data file for use with EPGImporter.
< pw.txt <-- A text file (stored locally) containing only your Schedules Direct password. The contents are fed to the command so you don't have to enter it manually. Needed when automating getting your guide data.

As I shared with iBoston, it took me a lot of researching to glean the various bits of info needed from different sources to get this working under Linux for my Edision, including some very crucial info he provided regarding xml file structure. We both agree that setting this up is not a simple endeavour but it is well worth it IMHO. In addition to the above you will also need to define custom.channels.xml and custom.sources.xml files using the service ids for your FTA channels in your receiver and the location where you store your downloaded guide data.

I started out by acquiring my guide data manually, once a week, typing the necessary commands into the terminal to download my data from Schedules Direct. I then ran the EPGImporter plugin on my Edision to get it into the receiver. If you are like me, you will probably get tired of forgetting to do this. You can automate the whole process by using a script and having it run as a cron job and setting the EPGImporter plugin to grab the data automatically. I use a NAS drive for storage and the scripts I wrote are run on a computer I keep on 24/7.

Along with program start and end times most of the Schedules Direct epg data includes episode titles and program descriptions. I am in my second year as a subscriber and can highly recommend the service they provide to hobbyists like us. If you have the skills and are willing to put some effort into using the data they supply it is well worth the time and cost to get a reliable, working guide on your FTA receiver.

 

[iBoston]

If your looking for EPG for Patient Channel (they update it every quarter year) although it rarely changes much... So, even if you update it once a year, it won't be off by much. :

Your healthcare growth partner from Discovery to Recovery.

WebMD Ignite helps health organizations grow and thrive, with exclusive solutions that drive measurable action throughout every stage of the healthcare journey.

www.thewellnessnetwork.net

I also included an XML file i created that breaks it down to something closer to what EPGimport would need.. This is a template my program uses to build the EPG. The txt file breaks it down to a weekly schedule. My program then utilizes the template xml file to build date/start/stop times to comply with XMLTV format.

 

[FTA4PA]

If your looking for EPG for Patient Channel (they update it every quarter year) although it rarely changes much... So, even if you update it once a year, it won't be off by much. :

Your healthcare growth partner from Discovery to Recovery.

WebMD Ignite helps health organizations grow and thrive, with exclusive solutions that drive measurable action throughout every stage of the healthcare journey.

www.thewellnessnetwork.net

I also included an XML file i created that breaks it down to something closer to what EPGimport would need.. This is a template my program uses to build the EPG. The txt file breaks it down to a weekly schedule. My program then utilizes the template xml file to build date/start/stop times to comply with XMLTV format.

Thanks!

 

[iBoston]

Ill wait a couple days, but it appears directv has taken down or moved there EPG system via web that allowed users of MakeXMLTV to create free EPG listings. I also was using directTV to get some of my channel guide information.

Can anyone else confirm this....

 

[iBoston]

Regarding the AWE - A wealth of Entertainment channel on 117w KU. Has anyone found an EPG that actually matches whats on? Even the AWE website doesn't appear to match whats actually 'Live'. I just watched Dream Cruises, and now its Travel in style, and there is NO pairing in the EPG that shows dream cruises and then Travel in style, and even seeing those shows separated on the page aren't even the same episode to whats actually 'Live'....

Sources I've thus tried :

International Schedule – AWE

www.awetv.com

A Wealth of Entertainment - TV Listings Guide

www.ontvtonight.com

Is this because its reported as AWE Encore/Plus? Which might be different? But, EPGs i find for that appear to be the exact same line up so far...

 

[cyberham]

I can't find a schedule that matches AWE Encore. We receive AWE on our Eastlink cable channel. Its schedule matches your International schedule. But it differs from AWE Encore. I suspect AWE Encore is a clever way the owners of AWE can sell the same product to distribution systems who offer AWE for free to viewers as they offer to pay TV systems.

And then...what is AWE Plus?

 

[Keith Brannen]

I never found a schedule for AWE Encore. Only thing I did find was that it seems it is basically a streaming channel. Could be that on some of those streaming systems there might be a schedule, but probably only accessible if you have that system.

 

[iBoston]

AWE Plus is the same thing as AWE encore. I believe i found a EPG for it...

Selling mega mansions is on now, and if travel in style is next re Mauritius, ill have found it. If it checks out, ill program it into my EPG software and get it live on my Enigma2 STB. I found what appears to be a matching EPG via AWE Plus | DistroTV Live Channel .... and by studying the html, found where they get it from, and below is the result of that, i was able to specify the number of hours to fetch, such as 24 or 48 hours etc...

*Update - It is a match. If i don't have EPG, i almost never watch the channel, so this will give this channel a chance...

 

[clucas]

I have it in my TitanTV.com schedule listed as AWE+

 

[iBoston]

Got it! - Now i need to do Taiwan Plus if i can...

 

[iBoston]

Darn, im getting quick with this.. Deciphering TawainPlus website for schedule data was easy as it was in json format, much like awe+. So, i just duplicated the AWE function i created , and made a few tweaks, and wala, I now have Taiwan +.

I now have full EPG again! Yea!!!

 

[iBoston]

This one was a bit tricky. This wasn't JSON, nicely formatted data, i had to decipher very non descriptive HTML. But, my Kung-Fu was strong.

Teleboy.ch has the wrong TIMES! which was nicely formatted in JSON.
TV listings – Wedo Big Stories - Teleboy (Times mismatched)

Had to get it directly from Wedo, but the nice thing - It supplies 5 days of EPG.
https://en-gb.wedotv.com/channels?program=wedobigdocs (Accurate)

I now have EPG data for Wedo Big Stories

The start times are bizarre (not aligned to top of hour) , so definitely need EPG data for this channel!

 

[Keith Brannen]

Teleboy.ch has the wrong TIMES! which was nicely formatted in JSON.
TV listings – Wedo Big Stories - Teleboy (Times mismatched)

Had to get it directly from Wedo, but the nice thing - It supplies 5 days of EPG.
https://en-gb.wedotv.com/channels?program=wedobigdocs (Accurate)

Interesting. For me, teleboy.ch times are accurate, and when I view the wedo site, I only get a screen with 2 hours of programming (could be my system/browser, who knows).

Glad you found something that works for you!

 

[iBoston]

Are you sure the minutes are correct for you? - The correct shows, in the correct order, but if you look at the times, they are not correct for me. They are off by 20+ minutes of actual start/stop times.

Regarding wedo site, yes - your correct, the viewable website only shows 2 hours of programming, but the html code within contains 5 days of EPG...

 

[Keith Brannen]

Screenshot of listings, only a minute different on a couple of the shows that you have listed:

 

[iBoston]

Hmm, maybe i screwed something up in my URL call. I did try to omit as much as i could , so, maybe i took something out that i needed.. I'm not gonna bother with it unless i have issues with the current.

I noticed one thing with the teleboy. It required an api-key or it wouldn't authorize. In the 24 hours i ran the script , the api-key never changed, but the key could change weekly etc, in which case, that would complicate things as then you would need to re-fetch the current api-key.

 

[iBoston]

They modified their website, which broke the EPG, but on the positive side, they provide more detailed information and better anchors. I suspect there will be additional changes, as their website is a bit buggy. I was able to fix it and get it working again.