Software Systems

[coliver988]

ok, let's see:

[FONT=arial,helvetica]There should be an encapsulating top level element here that
somehow designates the universe, milieu (sp?). Something like

<milieu date="1105-001" id="canon">[/FONT]

[FONT=arial,helvetica]
That would be better, although I think just the year would suffice. Sort of the Imperial survey year.

[/FONT]

[FONT=arial,helvetica]date should be an attribute and attached to any element that
needs it, inheriting downward.[/FONT]

[FONT=arial,helvetica]
Could be - but I'd just use the date per sector level I think, as above. I'm not big on attributes, but that could simply be inexperience with XML & XML processing.

[/FONT]

[FONT=arial,helvetica]As a matter of design style, I would make information that can't
conceivably have sub-elements be an attribute, rather than
an element. At least in the general case. Thus,

<system name="Efate" location="1705">[/FONT]

[FONT=arial,helvetica]
see above - it is easier to search elements for getting to a specific node; i.e., I use XPath to find the node I need and get everything at that level. You can do that with attributes versus elements, but seemed simpler/more efficient to use elements (again - there are entire non-Traveller holy wars on the entire attribute vs element thing). For my purposes in the initial development, elements made sense for where I was planning on going with this - the system name was the top-level node, so that I could get an entire node (and hence all the system data) in a single search via XPATH (well, except for the issue of identical names...)

[/FONT]

[FONT=arial,helvetica]Shouldn't "Political" be "allegiance" for consistency. At that,
we need some sort of canonical list of allegiance codes. The two
character codes seemed to be running out of space. Actually,
now that I think about it, there should be a series of "Imperial
Standards" which spell out these sort of things. Thus,
"Imperial Standard 2137 - Three Character Allegiance Codes (IS-2137)".
Q.V. ISO-3166-1 http://en.wikipedia.org/wiki/ISO_3166-1_alpha-3[/FONT]

[FONT=arial,helvetica]
Probably so - I may change that if I ever get time/inclination to start that project up again.

[/FONT]

[FONT=arial,helvetica]Isn't the UPP technically all of the three above? I would just
make it an attribute, probably a single one. At the point we
have to pick apart the upp to get at the specific individual numbers,
picking apart the string for the starport code and tech level isn't
really any more work.[/FONT]

[FONT=arial,helvetica]
technically, yes. but I was planning on expanding out the starport (another stalled project - real life tends to be more fun for the most part!). My plans included how busy in terms of traffic, cargo, people, possibly merchant info, stuff like that. And I've a copy of Grand Census, where tech can vary on application, so the idea was that I may expand out the tech level as well, sort of like:

<tech>A
    <medical>B</medical>
    <...>
</tech>

Permalink is for Joshua Bell's Traveller map - he's added significant new API links, so my plan is to replace that with the new one. The program, when parsing the file & loading the planet, had a button to load the map in your browser for that system if there was a permalink node (the beauty of optional XML stuff!)

Thanks for the suggestions - one of the reasons this stalled out was someone else was doing a system XML layout & I liked that one better. I'll probably use most of your suggestions if I get back to that program (which actually was in the middle of splitting out the XML handling into a class or namespace, so that I could re-use the same stuff for different things as well as add support for SEC files; then I realized I was essentially writing Heaven&Earth/Traveller Universe, and sort of stopped: they were switching to MS SQL-lite (or whatever MS calls their free SQL server) and started to think it would be significantly easier to use that as my backend, as this was a personal project. So I stalled...although there really should be some standardized data transport mechanism for subsectors: the SEC files, while good, are limited by virtue of being a fixed-width file. XML has the flexibility we need. So that's why I'm following this thread with interest, and threw out what I've done in the past for at least discussion points.

[/FONT][FONT=arial,helvetica]See, this is exactly what I wanted

One thing that putting a 'timeframe' into it would allow you to, for instance, watch the unfolding of the 5FW or the Imperial expansion as a kind of slideshow... which would be so awesome for us "history" fiends.

Exactly!
[/FONT]

 

[DonM]

This discussion is really outside of the T5 forum, so I will move the thread.

 

[Tysis]

Speaking as a programer I find XML handy in games for two things:

1. Data exchange
2. Storing data tables

The second item may seem odd but frankly most of the time I do not need the complexity of a database. I can implement some kind of data structure to parse the file into a internal list. With .NET you can treat an XML file just like a database if you like.

I do like to store any saved information in binary files for size and performance issues.

 

[robject]

This may seem trivial or silly to the software engineering community, but I've been taking a step back from full-XML-representations lately, as it tends to slow both me and my computer down.

So I've been using XML simply to bucketize my data, and left the intensive data processing for more Traveller-coupled code. This means I've ignored some of the infinite flexibility that XML provides, but as in all things such things are more easily added than taken away. And of course representation without implementation leads to the kitchen sink, especially in my non-work personal programming projects.

Anyhoo, my template has been the lowly SEC file, and my XML buckets simply serve to delimit the sections. There's plenty of room for appended data to any degree of complexity, but the bare essentials are both fast and easy for both humans and Traveller-centric code.

As an example, here's my XML-SEC file for Corridor, with just a sampling of the UWP data since that's all old hat.

<sec name="Corridor" milieu="Second Survey" desc="AOTI recapture">

<header>
Sector: Corridor Sector
Alt:    Sector eE
DOS:    corridor
Coord:  eE
Ref:    9920 Ring/Ray 62768
Gen:    Tue Apr 25 03:36:14 2006
</header>

<subsectors>
A: Khouth
B: Khukish
C: Lemish
D: The Narrows
E: Ian
F: Strand
G: Naadi
H: Uantil
I: Shush
J: The Empty Void
K: Atu'l
L: Kivu
M: Two Worlds
N: Ashishinipar
O: Sinta
P: Sashrakusha
</subsectors>

<notes>
This is pre-Rebellion data.
Bases converted to T5 (N,S,M,D,W) notation:
   C changes to S or M
   G and J change to N, usually
   H changes to S, N, W, or even NS

ref 0111 Shinku (MJ)  "Shinku" 
ref 0112 Kiran (MJ)  "Kiran" 
ref 0113 Aga Sugek (MJ)  "Aga Sugek" 
ref 0205 Koergfoes (MJ)  "Snapshots of the Occupation" 
ref 0213 Erlu (EA)  "Ian Subsector" 
ref 0311 Jubal (EA)  "Ian Subsector" 
ref 0312 Muugagen (MJ)  "Defying the Wolf" 
ref 0313 Yubitty (TD)  "Corridor" 
ref 0338 Ishirdu (TD)  "A Concise History of the Third Imperium" 
ref 0513 Kumorle (MJ)  "Kumorle" 
ref 0516 Raiga (EA)  "The Corridor Sector" 
ref 0611 Ikhur (MJ)  "Rapid Repo" 
ref 0614 Dywosik (MJ)  "Defying the Wolf" 
ref 0701 Gzorraeth (TD)  "Corridor" 
ref 0816 Antiquity (WBH)  "Survey and Exploration" 
[... lots more references clipped for brevity ...]
</notes>

<data>
Tersta         0102 C474522-7    Ag Ni An                320 Im F3 V 
Khouth         0104 A8C3999-D    Hi Fl Cp                420 Im M3 V 
Ankirst        0105 C356112-9    Lo Ni                   421 Im M1 V K3 VI 
Ofo-Nebus      0106 B541488-8 N  C1 Ni Po                701 Im M7 V M0 D 
Synez          0109 E864256-7    Lo Ni                   620 Im M2 V 
Koergfoes      0205 B54359A-B N  Ni Po Rs                322 Im M5 V 
Mikesh         0206 C8B7ACB-E S  Hi Fl                   625 Im K1 V 
Pergzitt       0209 B625354-B    Lo Ni                   311 Im M6 III K5 V 
Dry            0210 C110877-B S  Na                      801 Im M5 V M9 V 
Serk           0304 B89A866-C N  Wa                      723 Im M4 V 
[...etc etc...]
</data>
</sec>

 

[Tysis]

Large XML files take an annoyingly noticeable bit of time to process. I tend to use XML files for data when I do not need the complexity of a database.

I store my data in binary files, which are much smaller and faster.

To speed up access of XML data files you can pre-load the ones that you use the most.

 

[Clay]

The slow processing time of XML files is a function of your processing engine more than XML itself. Engines such as MSXML and Xerces are very slow. I've been using libxml2 for a while now and it is quite fast. I did my comparison using insurance claim and payment data, where the amount of data can be fairly overwhelming.

Don't forget that XML gives us a lot of interoperability, and eliminates the need to write our own parsers. That's a lot of time taken off development for what is almost certain to be a side, non-paying project.

 

[robject]

Don't forget that XML gives us a lot of interoperability, and eliminates the need to write our own parsers. That's a lot of time taken off development for what is almost certain to be a side, non-paying project.

Agreed: sharing data structures is nice.

I suggest that anyone with an application in mind, however, should not wait first for a standard. Standards themselves are very time-consuming, side, non-paying projects as well, especially for hobbies.

 

[Tysis]

Agreed: sharing data structures is nice.

I suggest that anyone with an application in mind, however, should not wait first for a standard. Standards themselves are very time-consuming, side, non-paying projects as well, especially for hobbies.

I take it you have been on a standards committee or two yourself? I have been on a couple in the financial industry and I can tell you he is right on. They seem to consume time and after a while your employer wants you to "donate" some time to it.

And yes, MSXML is a bit slow (I have used it directly and I am assume this is what .NET uses). The .NET technologies seem to speed up quite a bit after the first load. In general this is not a real problem for anything hobby related. With the .NET technologies you can also switch between writing data to binary and XML files fairly quickly.

 

[whartung]

Agreed: sharing data structures is nice.

I suggest that anyone with an application in mind, however, should not wait first for a standard. Standards themselves are very time-consuming, side, non-paying projects as well, especially for hobbies.

Also, no one is suggesting these formats for internal processing, rather simply for data interchange. Having an, *ahem*, "canonical" interchange format makes your internal choices as a developer irrelevant to the user community as long as your software supports importing and exporting data to the common standard.

It does bring up a couple of issues that may be worth discussing.

Is there a way to make the format reasonably portable across versions? Obviously later versions, perhaps, add more data, but we already have the constraints of the earlier versions of the game, does anyone think it impractical to have a format that spans the games in a reasonable way?

Star data is the most portable, as it's changed the least. I don't know whether character data is viable, unless there are mechanisms that can be used to convert, say, a T20 character to a TNE character.

I don't know if anything can be done with starships.

What about an extended "notes" field along with everything else? Doesn't need to be really involved, but I think something that allows someone to make some notes and then import/export that data would be valuable.

Some might have more detail, or perhaps even structured notes, for their systems or characters, but that would be outside of the scope of a simple notes field I think.

Even "name", "summary", "description" would be valuable.

 

[coliver988]

Also, no one is suggesting these formats for internal processing, rather simply for data interchange. Having an, *ahem*, "canonical" interchange format makes your internal choices as a developer irrelevant to the user community as long as your software supports importing and exporting data to the common standard.

It does bring up a couple of issues that may be worth discussing.

Is there a way to make the format reasonably portable across versions? Obviously later versions, perhaps, add more data, but we already have the constraints of the earlier versions of the game, does anyone think it impractical to have a format that spans the games in a reasonable way?

Star data is the most portable, as it's changed the least. I don't know whether character data is viable, unless there are mechanisms that can be used to convert, say, a T20 character to a TNE character.

I don't know if anything can be done with starships.

What about an extended "notes" field along with everything else? Doesn't need to be really involved, but I think something that allows someone to make some notes and then import/export that data would be valuable.

Some might have more detail, or perhaps even structured notes, for their systems or characters, but that would be outside of the scope of a simple notes field I think.

Even "name", "summary", "description" would be valuable.

That's why the general consensus has been for an XML format. Not that I really expect any standard until someone just does something & says 'Hey, here's how I'm doing it' and others simply start using that by default, embellishing for their own systems (probably how the SEC files got done - someone decided to make a file to handle stuff for some software they did, and everyone else just started using that since the files were there for use; de facto standard by usage [kinda how MS Word is a standard, albeit a proprietary one). But as previously mentioned, with XML as a transport layer, you can pick and choose the stuff you are actually using, or even add your own extensions. Web browsers display XML as a tree by default, so that you can collapse layers and stuff. So anyone could view an XML file in their browser readily.

As for handling multiple versions, one method could be:

....
  <System Name="Regina">
      <Version="Classic">
        <UPP>788899</UPP>
        <Starport>A</Starport>
        <Tech>A</Tech>
           ....
      <Version="GURPS">
        <UPP>788899</UPP>
        <Starport>V</Starport>
        <Tech>9</Tech>
          ....

(I separate out the tech & starport currently 'cause I want to expand out the 'port info (high/low components, traffic, stuff like that; ditto for tech: I want to break the tech out as per World Builders/Grand Census. But with XML, those who don't need/want that level simply ignore it when loading the data. That and to make my example a bit more clear).

Notes and things would be similar: there could be a top-level notes field, but enterprising users could add extensions (GM notes vs player notes, for instance, so that you coul dhave 1 data file that was used by both, but have a GM or player mode for the reader).

As also previously mentioned: stuff done by committee usually manages to match no one's expectations or actual needs...so I really expect a few people to start releasing things and eventually a standard will coelesce around that (and then I'll rewrite my XML stuff to handle that format rather than the one I originally came up with (which I may rewrite after some excellent suggestions in this thread (and if anyone knows LISP, it seems my writing style uses too many parenthesis as well!)))

 

[Norman King]

Why reinvent the wheel?

PCGen already has file formats already created:
http://pcgen.sourceforge.net/

I am in the T20 mods Yahoo Group and I was testing out the Traveller T20 abilities.

You can join it at:
http://groups.yahoo.com/group/T20_PCGEN

I think they tried to add in Mega Traveller abilities into it, since MT is based on CT, and T5 is going back to CT/MT rules and the D6 system, you might be able to compare notes with the PCGen programmers to see how they are doing it.

I think having a Java based PC Generator is a great idea, because I've seen a lot of DOS, Unix, Windows, Mac etc only versions of various Traveller software with very few ports between the systems.

They use LST files but are looking to use XML files as well.

 

[whartung]

PCGen already has file formats already created:
http://pcgen.sourceforge.net/

I am in the T20 mods Yahoo Group and I was testing out the Traveller T20 abilities.

You can join it at:
http://groups.yahoo.com/group/T20_PCGEN

I think they tried to add in Mega Traveller abilities into it, since MT is based on CT, and T5 is going back to CT/MT rules and the D6 system, you might be able to compare notes with the PCGen programmers to see how they are doing it.

I think having a Java based PC Generator is a great idea, because I've seen a lot of DOS, Unix, Windows, Mac etc only versions of various Traveller software with very few ports between the systems.

They use LST files but are looking to use XML files as well.

Noble, but PCGen is focused solely on a) char gen and b) multiple systems, so I wouldn't necessarily rely on them as a reference for a traveller schema model.

If they adopt XML, then it should (ideally) straightforward to convert from the STX format (Standard Traveller XML) to the PCGen XML format (say, via XSLT...), and perhaps be able to transfer back.

 

[coliver988]

Why reinvent the wheel

C'mon - this is Traveller: not only do we reinvent the wheel with each version released, but we also make multiple tweaked versions of the same wheel for each version of Traveller!

(On a more serious note: possibly because what is out there won't do what some people may want it to do. That and if you know programmers, you know that each one has a better way of doing the same thing!*)

*note: IRAAP (I Really Am A Programmer). Go figure...

 

[Tysis]

C'mon - this is Traveller: not only do we reinvent the wheel with each version released, but we also make multiple tweaked versions of the same wheel for each version of Traveller!

(On a more serious note: possibly because what is out there won't do what some people may want it to do. That and if you know programmers, you know that each one has a better way of doing the same thing!*)

*note: IRAAP (I Really Am A Programmer). Go figure...

I am forced to agree. On both counts.

Also a programmer here...

 

[robject]

Also, no one is suggesting these formats for internal processing, rather simply for data interchange. Having an, *ahem*, "canonical" interchange format makes your internal choices as a developer irrelevant to the user community as long as your software supports importing and exporting data to the common standard.

Standards are always external... and they are always difficult. I'd suggest that the time spent on inventing a vapor-standard is better spent on writing code that does something, then tacking on a "Save File As..." option to dump to as many TXML standards show up.

There are two -- no three -- other reasons I say this:

(1) There's very little Traveller freeware out there, so supporting multiple standards is less imposing than one might think.

(2) Developers are likely to coalesce around two or three superior, competing standards, rather than N standards for N programs. The time spent supporting two or three standards is very little extra work, especially compared to the time spent designing a standard by committee. And in fact the community may be better served back-implementing a standard after some magic number of already-working applications (how about two?) agree on a common interchange format.

(3) Standards were made to be broken. As soon as a standard is out, it's obsolete. Thus the support burden is not removed by a standard. It's not really even simplified by the presence of a standard.


Boy do I sound contrarian.

The myth that a standard will promote development or make it easier is just plain false.

On the other hand, it's very useful to itemize the data elements that one might expect to see dumped from these various tools. That alone can suggest a small number of formats, and is probably sufficient to prime the creative energies of our developer base.


Sorry to sound so negative, but I'd like to see less fretting and more coding. Create a reason for a standard first.

 

[robject]

[...]2) I have bad experiences with some of the public Traveller communities (and various gamer forums in general,) so I am reluctant to bring this up elsewhere [...]

Well, I hope I'm not being too much of a downer, and I don't want to make this a bad experience for you. I feel somewhat concerned that I'm hindering more than helping.

It would be nice to be given an XML file and just slurp in character data from it....

James has the super-buff XML, whereas I'm pretty plain with mine. I'd do a character like this:

<character>

   <name>Fred</name>
   <title>Ex-Scout Captain</title>
   
   <details>   
      <birthdate>110-980</birthdate>
      <age>37</age>
      <home>Regina/1910 Spinward Marches</home>
      <vitals>6'0", 180lbs</vitals>
   </details>
   
   <upp>
      <gen>SDEIES</gen>
      <c1>7</c1>
      <c2>8</c2>
      <c3>9</c3>
      <c4>8</c4>
      <c5>7</c5>
      <c6>6</c6>
   </upp>
   
   <history>
      <scouts>
         <terms>4</terms>
         <events> 
            [... room for skill acquisition etc here ...] 
         </events>
      </scouts>
      <marines>
        ...etc etc...
      </marines>
   </history>
   
   <posessions>
      <credits>50000</credits>
      <weapon type="ACR-10">
         <qrebs>
            <r>-1</r>
         </qrebs>
      </weapon>
   </posessions>
   
   <skills>
      <engineer>3</engineer>
      <driver>2</driver>
   </skills>
   
</character>

 

[robject]

My ship data. I haven't put thought into the sensor data.

There's plenty of room within the components to include QREBS data.

<ship>
   <name>Banshee</name>
   <type>TS</type>
   <class>Pocket Trader</class>
   <details>
      <builder>Baraccai Technum</builder>
      <keel_laid_down>200-1095</keel_laid_down>
      <production_number>40126</production_number>
   </details>
   <hull type="A">
      <frame>streamlined</frame>
      <gear>L</gear>
   </hull>
   <bridge>
      <volume>10</volume>
      <model>3</model>
      <consoles>
         <sensor> ... </sensor>
         <sensor> ... </sensor>
         <sensor> ... </sensor>
      </consoles>
   </bridge>
   <jump type="A" />
   <maneuver type="A">
      <qrebs>
         <s>2</s>
      </qrebs>
   </maneuver>
   <power type="A" />
   <fuel>22</fuel>
   <staterooms>6</staterooms>
   <low>6</low>
   <turret type="Hybrid-LMS" />
   <cargo>25</cargo>
</ship>