Page 1 of 1
| Possible new file formats for timetables 19/04/2010 at 12:28 #1143 | |
GeoffM
6282 posts |
The current file format for timetables is a custom, binary format. With future expansion in mind, I am considering changing the format to one that firstly supports expansion and secondly is easier for 3rd party tools to manipulate. Existing timetables will still be supported. With that in mind, please list any pros or cons against the following:
Or suggest another format with the reasons why, for and against. Please note that this thread isn't about ideas and features to add to timetables - those can continue to go in the Feature Requests forum and they'll be considered there. Thanks. SimSig Boss Log in to reply |
| Possible new file formats for timetables 19/04/2010 at 12:28 #8630 | |
|
GeoffM
6282 posts |
The current file format for timetables is a custom, binary format. With future expansion in mind, I am considering changing the format to one that firstly supports expansion and secondly is easier for 3rd party tools to manipulate. Existing timetables will still be supported. With that in mind, please list any pros or cons against the following:
Or suggest another format with the reasons why, for and against. Please note that this thread isn't about ideas and features to add to timetables - those can continue to go in the Feature Requests forum and they'll be considered there. Thanks. SimSig Boss Log in to reply |
| Possible new file formats for timetables 19/04/2010 at 17:22 #8639 | |
JamesN
1575 posts 
|
Well, I've been loosely involved in the development of a new bit of Timetabling Software which makes extensive use of XML in it's inputs. But it outputs to Clive's Convdata format, so perhaps direct support for either of those two would be good.
Log in to reply |
| Possible new file formats for timetables 19/04/2010 at 19:44 #8649 | |
|
Adrian the Rock
111 posts |
I would have thought XML the obvious standard going forward. It's the de facto standard for interfacing files within the IT industry and their is plenty of software (free or otherwise) available for manipulating XML. For some months now I have even been having vague thoughts of suggesting a converter from Convdata format to/from XML myself (the main thing that has held me back is not really having time to get stuck into doing anything like this myself). To give a trite example, on a recent project at work I needed to load some XML data into an Access database (but without using its out-of-the-box import facility, which was far too simplistic). When I got down to it, it took me just two lines of VBA code to get the file read in, parsed, and loaded into an in-memory tree structure (using the MSXML package) that I could then traverse and process as I wanted using easy function calls. The entire code I wrote - using the contents to to populate eight database tables with about ten attributes on each - can't have been more than about 200 lines. XML is very good for structured (as opposed to "normalised"  data, of which SimSig TTs are largely made. For example a train has an ordered array of locations, each of which has various attributes including an ordered array of actions.Another advantage is that you can extend the schema subsequently with very limited impact on programs written to the old version, or when new programs read older files. Say you added a new train location attribute such as "stop only if the signal's red" (eg for freight trains booked to be looped). This would just be a new XML sub-tag within the <location>. Existing programs using the file would almost certainly still work - they just wouldn't "see" (or output) the new tag. If a new program read in an old file (without the tag), it would just behave as though the tag had been set to a null value in the new format within every <location> in the old file. Log in to reply |
| Possible new file formats for timetables 19/04/2010 at 21:47 #8654 | |
|
Helper
20 posts |
I support XML for the reasons stated above. It can easily be imported to Excel in seconds or to many free loaders that enable TT data to be manipulated without going to the expense of buying software. The only other format I would suggest, perhaps to keep it even simpler is CSV. Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 00:38 #8658 | |
|
lpeters
160 posts |
.csv is already supported; however, I do seem to have problems with only half the timetable converting; always seems to miss out half of the timing points.
Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 07:16 #8659 | |
Chrisrail
384 posts |
The existing format is fine as it can easily import data from another source (Not that I know anything about that !!!)
Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 12:26 #8664 | |
|
bill_gensheet
1316 posts |
As long as the following two functions remain available, I don't think I have a problem with file format: The sim has a similar input and validation behaviour to that currently. There is a human comprehensible format much as we see in convdata text files now where editing can be done in text editors / word processors. I would not be so keen on a format that needed (as opposed to offered for those who want it) any programming input to use it. Bill Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 13:28 #8667 | |
|
clive
2738 posts |
bill_gensheet said:The sim has a similar input and validation behaviour to that currently. There are no plans to change the behaviour of the sim itself, just what a .WTT file looks like if you dig into it. bill_gensheet said: There is a human comprehensible format much as we see in convdata text files now where editing can be done in text editors / word processors. XML files are plain text, but opinions vary on how easy they are to edit using text editors. Various specialized XML editors exist, but I've never used them so I can't comment on how easy they are. lpeters said: .csv is already supported The csv import/export functions are not particularly great. In particular, they can't handle some of the more esoteric features of timetables at present, let alone anything that might come along in the future. Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 18:43 #8683 | |
|
Sacro
1171 posts |
Right then, here goes, I am the person that JamesN mentioned was developing alternate software for timetable creation and editing/viewing and here's my take on it... I am for the idea of a new timetable format, and I'd personally stick a vote in for XML, it's very suitable for nested data (unlike csv which can only do row/column), however, if this is change for changes sake then it might not be as great an idea, I have coded things that read and write WTT files, that read and write XML and that write to Convdata. I've posted a bit of sample XML and the corresponding ConvData, perhaps these might be of use for a possible format, I'd appreciate thoughts/ideas and when I get change I'm going to perhaps rattle up some kind of UI for viewing/editing, I'm still honing the XML format and would be glad if people made suggestions / improvements. @Chrisrail, I am aware of the source and tool of which you speak and I have written my own tool which is roughly 10x faster than the one you use (on my machine), outputs to XML and thence to Convdata (which obviously can then go onto WTT), and (apparently) has less 'quirks' when it comes to sifting through the data, also takes XML input files which I have done for most sims and are easily created, if we do go ahead with a new timetable format I will be happy to release it to you. Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 22:11 #8691 | |
|
GeoffM
6282 posts |
Sacro said:however, if this is change for changes sake then it might not be as great an ideaThere are some enhancements that culd be added which would "break" the current binary format, so this is using the opportunity to roll a few things in together. Sacro said: I've posted a bit of sample XMLInteresting to see your take on it, particularly naming conventions and lists of items, both of which have varying schools of thought. For example, I've seen XML with camelCasing, PascalCasing, Hyphenated-words, ALLUPPERCASE, alllowercase, and combinations thereof. SimSig Boss Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 22:20 #8693 | |
|
Sacro
1171 posts |
Well if you're going to be superseding the current binary format then perhaps moving to an XML based system might be a nice idea. Casing was originally PascalCase for the outputted xml yet lowercase for the input xml used to specify tiplocs/entry poiints/other rules, this got rather complex (funnily enough!) so I switched it all to lowercase, this could be changed with relative ease however. Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 22:36 #8695 | |
|
GeoffM
6282 posts |
My preferred usage is camelCase or PascalCase for readability with multiple words. I don't like abbreviations so your <frt> is out! :P I'm about to begin documenting what the data currently consists of in a DTD (or preferably W3C XML schema if supported with my tools). I'll reply with the URL when I've gotten somewhere with it. SimSig Boss Log in to reply |
| Possible new file formats for timetables 20/04/2010 at 22:41 #8696 | |
|
Sacro
1171 posts |
Ahh, I was using some of the three letter abbreviations from Convdata! A DTD or XSD would be nice, then it gives a nice standard to work against.
Log in to reply |
| Possible new file formats for timetables 12/07/2010 at 18:52 #10053 | |
|
GeoffM
6282 posts |
I've not done anything on this for several weeks now. Instead, for now I've put the work so far into the Wiki - see http://www.SimSig.co.uk/dokuwiki/doku.php?id=usertrack:ttuse:xml_timetable Please keep discussion in this thread. SimSig Boss Log in to reply |
| Possible new file formats for timetables 12/07/2010 at 21:56 #10056 | |
|
Sacro
1171 posts |
Hmmm, not bad, would prefer electrifcation to be xs:boolean (unless you have some sneaky plan?), also perhaps drop the prefix from IDs, <schedule scheduleID="1A01"> would be tidyer as <schedule ID="1A01">, "bonusSocring" is misspelt. It's looking decent though! Log in to reply |
| Possible new file formats for timetables 13/07/2010 at 19:34 #10065 | |
|
Jan
889 posts |
I think the sneaky plan with electrification is that type="electrification" refers to the complexType definition right above, i.e. you can then use overhead25kvac, ThirdRail750dc, FourthRail750dc or diesel as values. Is there any particular reason why a similar scheme is not being used for accelerationBrakingProfile, too? Also, an element specifying the simulation version the timetable was created with could be useful, in case there have been breaking changes in the simulation data. Two million people attempt to use Birmingham's magnificent rail network every year, with just over a million of them managing to get further than Smethwick. Log in to reply |
| Possible new file formats for timetables 14/07/2010 at 08:13 #10067 | |
|
GeoffM
6282 posts |
Nothing sneaky, probably just an oversight on my part. I think each of the:<xs:element name="overhead25kvac" minOccurs="0"/>should be <xs:element name="overhead25kvac" type="xs:boolean" minOccurs="0"/>accelerationBrakingProfile is missing. If it were to use the current method then it would be a simple integer, 0 to 4, but to cope with future expansion I think I intended it to be a complex type but forgot to put it in - future expansion in this case being the ability to specify your own characteristics and acceleration/braking curves. Version number is possible of course, but using XML shouldn't mean any changes break things. In fact, it should have been there anyway (currently 2), also an omission. Feel free to edit obvious errors. Please try filling in the timetable to see if the structure makes sense. Regarding id versus scheduleID, I don't mind either way. Generally I like to be more verbose and obvious but I guess in this case it's pretty obvious what the ID relates to and thus doesn't need expansion. SimSig Boss Log in to reply |
| Possible new file formats for timetables 14/07/2010 at 09:08 #10069 | |
Right Away
71 posts |
I wonder if the ‘electrification’ complex type, with its allowed value of ‘diesel’, might be more appropriately named ‘motivePower’? Should elements ‘ThirdRail750dc’ and ‘FourthRail750dc’ be ‘thirdRail750dc’ and ‘fourthRail750dc’ respectively, i.e., lower case initial capital? Perhaps in the trainCategory complex type the ‘electrification’ or ‘motivePower’ element should have maxOccurs specified? The default is 1 but I wonder if we should set this higher so that dual voltage trains, like the class 378s on London Overground, can be specified as both overhead25kvac and thirdRail750dc?. A train specified thus might incur a small time delay when it halts at the point where it is switched from one power source to another or perhaps be subject to delay if one or both power sources are disrupted. Log in to reply |
| Possible new file formats for timetables 16/07/2010 at 17:50 #10104 | |
|
Jan
889 posts |
GeoffM said:Nothing sneaky, probably just an oversight on my part. I think each of the: Doh, I completely overlooked that. Just forget that part of my posting. Another nice feature, if it hasn't been mentioned somewhere already, would be the possibility to preset some of the simulation launch options, especially the era setting. I'd still allow changing the settings by the user, but if you don't do anything, the correct options, i.e. mainly the era, and perhaps a specific scenario, are already selected. Two million people attempt to use Birmingham's magnificent rail network every year, with just over a million of them managing to get further than Smethwick. Log in to reply |