Trade Classifications

[jgreely]

It's the difference between a database and a report, basically. Historically, the SEC file seems to have served as both, with the fixed field lengths imposing some annoying limitations on the name and trade/notes fields. Stripping out the trade codes increases the amount of other data you can store without inventing a new file format, and you can always regenerate them for printing.

That said, if someone were to create a new file format that stored all of the useful system/sector information in a consistent way, while remaining human-editable (which rules out XML...), I would cheerfully add support for it in my upcoming rewrite of sec2pdf.

-j

 

[jgreely]

It's the difference between a database and a report, basically. Historically, the SEC file seems to have served as both, with the fixed field lengths imposing some annoying limitations on the name and trade/notes fields. Stripping out the trade codes increases the amount of other data you can store without inventing a new file format, and you can always regenerate them for printing.

That said, if someone were to create a new file format that stored all of the useful system/sector information in a consistent way, while remaining human-editable (which rules out XML...), I would cheerfully add support for it in my upcoming rewrite of sec2pdf.

-j

 

[Falkayn]

Originally posted by J Greely:
That said, if someone were to create a new file format that stored all of the useful system/sector information in a consistent way, while remaining human-editable (which rules out XML...), I would cheerfully add support for it in my upcoming rewrite of sec2pdf.

Gee, but XML is so nice ...

I think the best option would be a file format that can be easily edited using commonly available tools, but that allowed easy manipulation by computer programs. That does sound a lot like XML now I think about it.

It could be a CSV file format, and then I could write an Excel front end for it with macros to spit out the CSV.

Or it could be XML ...

My take on it would be that XML is only sensible if you want to represent relationships, and seeing as there is at least the set of Domain-Sector-Subsector-System relationships (without throwing in orbital bodies below system) XML would probably make a lot of sense.

It would be nice in that you could have more than one sector's info in one file, or even just one subsector's info. The hassle is having a program that could write it out nicely ... seeing as Microsoft's InfoPath is not yet available without the InfoPath client that leaves something reasonably common like an EXE (OS specific), Java ( ) or an Excel spreadsheet (again, mostly OS specific).

If we could agree on the format, I could probably whip something up in .NET or Excel for us to use.

 

[Falkayn]

Originally posted by J Greely:
That said, if someone were to create a new file format that stored all of the useful system/sector information in a consistent way, while remaining human-editable (which rules out XML...), I would cheerfully add support for it in my upcoming rewrite of sec2pdf.

Gee, but XML is so nice ...

I think the best option would be a file format that can be easily edited using commonly available tools, but that allowed easy manipulation by computer programs. That does sound a lot like XML now I think about it.

It could be a CSV file format, and then I could write an Excel front end for it with macros to spit out the CSV.

Or it could be XML ...

My take on it would be that XML is only sensible if you want to represent relationships, and seeing as there is at least the set of Domain-Sector-Subsector-System relationships (without throwing in orbital bodies below system) XML would probably make a lot of sense.

It would be nice in that you could have more than one sector's info in one file, or even just one subsector's info. The hassle is having a program that could write it out nicely ... seeing as Microsoft's InfoPath is not yet available without the InfoPath client that leaves something reasonably common like an EXE (OS specific), Java ( ) or an Excel spreadsheet (again, mostly OS specific).

If we could agree on the format, I could probably whip something up in .NET or Excel for us to use.

 

[jgreely]

XML and CSV are both useful formats for getting data out of one program and into another. They both suck to write by hand. CSV because you have to count commas to make sure your data ends up in the right field, and XML because parsing libraries abort on the first syntax error (and, in fact, are required to do so).

If I were designing a file format (let's call it .MTU, which is both familiar, and as far as I can tell unused elsewhere), it would look a lot like the MSEC format I threw together for sec2pdf. Simple keyword/value pairs, free use of white space to enhance readability, indentation for continuation lines, #-comments, etc.

Essentially, someone who just wanted a basic sector map could type in a single line for each world, containing Hex, UWP, PBG, Allegiance, and Name (in that order, so you could split on whitespace and let the name contain as many words as you want). I'd abandon the single-character base codes, and use continuation lines for Base, Trade, Code, Zone, and Star (one per line, indented to make it clear they're part of that system).

Each record could be followed by an optional multiline text field that started with a line containing the word "Description" and ended with a line containing only a period (this format may be familiar to some of you...).

Sector-level data would be done similarly, and applies to all system records that come after it, up until the next sector record. Ditto for sub-sector data if you're tracking that, and you could even extend it to domains. My routes and regions drop right in, and all it takes to add a new type of data is to come up with a keyword for it. Parsers that don't know about your new keyword just ignore it.

-j

 

[jgreely]

XML and CSV are both useful formats for getting data out of one program and into another. They both suck to write by hand. CSV because you have to count commas to make sure your data ends up in the right field, and XML because parsing libraries abort on the first syntax error (and, in fact, are required to do so).

If I were designing a file format (let's call it .MTU, which is both familiar, and as far as I can tell unused elsewhere), it would look a lot like the MSEC format I threw together for sec2pdf. Simple keyword/value pairs, free use of white space to enhance readability, indentation for continuation lines, #-comments, etc.

Essentially, someone who just wanted a basic sector map could type in a single line for each world, containing Hex, UWP, PBG, Allegiance, and Name (in that order, so you could split on whitespace and let the name contain as many words as you want). I'd abandon the single-character base codes, and use continuation lines for Base, Trade, Code, Zone, and Star (one per line, indented to make it clear they're part of that system).

Each record could be followed by an optional multiline text field that started with a line containing the word "Description" and ended with a line containing only a period (this format may be familiar to some of you...).

Sector-level data would be done similarly, and applies to all system records that come after it, up until the next sector record. Ditto for sub-sector data if you're tracking that, and you could even extend it to domains. My routes and regions drop right in, and all it takes to add a new type of data is to come up with a keyword for it. Parsers that don't know about your new keyword just ignore it.

-j

 

[Malenfant]

I think there should be some room for expandability in a data format too (in this case, extending the UWP codes, if necessary).

 

[Malenfant]

I think there should be some room for expandability in a data format too (in this case, extending the UWP codes, if necessary).

 

[Flynn]

That's an interesting idea, Mr Greely, but I believe if you did that, you'd also have to provide a parser that reads *.sec files or their equivalent, to translate the available data into the new format, and back again, for those that use Galactic and similar programs.

Two creds and all that,
Flynn

 

[Flynn]

That's an interesting idea, Mr Greely, but I believe if you did that, you'd also have to provide a parser that reads *.sec files or their equivalent, to translate the available data into the new format, and back again, for those that use Galactic and similar programs.

Two creds and all that,
Flynn

 

[RobertFisher]

Originally posted by Claire Rand:
can't help but thinking that since trade classes are totally dependent on world data, with no random element in them the whole trade class section is redundant..

Certainly. If I were designing top-down a comprehensive Traveller universe management system, I'd do exactly that.

Instead, I'm combining existing tools--both general & Traveller specific--with custom code to generate & manipulate existing & ad hoc data formats on an as-needed (or--more accurately--per-whim) basis.

Or, you could just say that my database & my report just happen to have the same format. My trade class script has a flag to generate either Bk3 or Bk7 classes. So, from my original "database" sec file, I can generate a Bk3 report sec file or a Bk7 report sec file.

In a similar vein, I'm personally committed to evolving file formats ad hoc rather than attempting a top-down design, though I'd contribute what I could to such an effort.

FWIW, any MS-Win or .net code would be virtually useless to me.

Sec2pdf is very nice though, Greely. I may eventually send you a patch to report malformed (or misread) sec files. (The silent failures & disappearing worlds from my failure to build a good sec file has bitten me a few times.

)

One thing that I've been thinking would be really nice would be a viewer with a query facility. Like where you could say, "Hilight all the worlds with tech level E+ & law level 4-."

I've faked that a couple of times with search & replace in VIM, amber zones for highilighting, & sec2pdf for viewing.

Eventually, though, I'm going to assign a real amber zone, & then maybe I'll have to work on something interactive.

 

[RobertFisher]

Originally posted by Claire Rand:
can't help but thinking that since trade classes are totally dependent on world data, with no random element in them the whole trade class section is redundant..

Certainly. If I were designing top-down a comprehensive Traveller universe management system, I'd do exactly that.

Instead, I'm combining existing tools--both general & Traveller specific--with custom code to generate & manipulate existing & ad hoc data formats on an as-needed (or--more accurately--per-whim) basis.

Or, you could just say that my database & my report just happen to have the same format. My trade class script has a flag to generate either Bk3 or Bk7 classes. So, from my original "database" sec file, I can generate a Bk3 report sec file or a Bk7 report sec file.

In a similar vein, I'm personally committed to evolving file formats ad hoc rather than attempting a top-down design, though I'd contribute what I could to such an effort.

FWIW, any MS-Win or .net code would be virtually useless to me.

Sec2pdf is very nice though, Greely. I may eventually send you a patch to report malformed (or misread) sec files. (The silent failures & disappearing worlds from my failure to build a good sec file has bitten me a few times.

)

One thing that I've been thinking would be really nice would be a viewer with a query facility. Like where you could say, "Hilight all the worlds with tech level E+ & law level 4-."

I've faked that a couple of times with search & replace in VIM, amber zones for highilighting, & sec2pdf for viewing.

Eventually, though, I'm going to assign a real amber zone, & then maybe I'll have to work on something interactive.

 

[RobertFisher]

Originally posted by Flynn:
Mind sharing your PDF or even just your *.sec file? I'd love to see what you're looking at, if it's cool with you.

Might even have some feedback for you, if you're interested in outside opinions,
Flynn

Sure. I'll try to remember to throw them up on my web site & email you a link. Here's hoping that I manage to steal some time from the holiday to work on MTU.

 

[RobertFisher]

Originally posted by Flynn:
Mind sharing your PDF or even just your *.sec file? I'd love to see what you're looking at, if it's cool with you.

Might even have some feedback for you, if you're interested in outside opinions,
Flynn

Sure. I'll try to remember to throw them up on my web site & email you a link. Here's hoping that I manage to steal some time from the holiday to work on MTU.

 

[jgreely]

Originally posted by RobertFisher:
Sec2pdf is very nice though, Greely. I may eventually send you a patch to report malformed (or misread) sec files. (The silent failures & disappearing worlds from my failure to build a good sec file has bitten me a few times.

)

This was one of the many areas where the "afternoon hack" nature of the script was starting to show through. I'm going to write a better parser for the rewrite, and clean up the interface so other file formats can be supported easily.

-j

 

[jgreely]

Originally posted by RobertFisher:
Sec2pdf is very nice though, Greely. I may eventually send you a patch to report malformed (or misread) sec files. (The silent failures & disappearing worlds from my failure to build a good sec file has bitten me a few times.

)

This was one of the many areas where the "afternoon hack" nature of the script was starting to show through. I'm going to write a better parser for the rewrite, and clean up the interface so other file formats can be supported easily.

-j

 

[FlightCommanderSolitude]

Originally posted by J Greely:
... XML and CSV are both useful formats for getting data out of one program and into another. They both suck to write by hand. CSV because you have to count commas to make sure your data ends up in the right field, and XML because parsing libraries abort on the first syntax error ...

There was a brief discussion of alternate formats here if you are interested. Your point about XML fragility is well taken, although one could argue it's a feature for a dataset to be well-formed.

Regarding trade classification and remark codes, I was wondering about that as well: since the codes are almost all algorithmic in the first place, why store them? Granted, some are not data-derivable, eg "subsector capitol". But the rest, it seems, could/should be left out of datasets and generated on the fly.

 

[FlightCommanderSolitude]

Originally posted by J Greely:
... XML and CSV are both useful formats for getting data out of one program and into another. They both suck to write by hand. CSV because you have to count commas to make sure your data ends up in the right field, and XML because parsing libraries abort on the first syntax error ...

There was a brief discussion of alternate formats here if you are interested. Your point about XML fragility is well taken, although one could argue it's a feature for a dataset to be well-formed.

Regarding trade classification and remark codes, I was wondering about that as well: since the codes are almost all algorithmic in the first place, why store them? Granted, some are not data-derivable, eg "subsector capitol". But the rest, it seems, could/should be left out of datasets and generated on the fly.

 

[clair_rand]

id say wahtever format ya go for nail it to the mast, or make extensions easy. and publish the lot, that way any halfwit can use it (thinks me), and write stuff to either generate it of process from it someway

 

[clair_rand]

id say wahtever format ya go for nail it to the mast, or make extensions easy. and publish the lot, that way any halfwit can use it (thinks me), and write stuff to either generate it of process from it someway