• Welcome to the new COTI server. We've moved the Citizens to a new server. Please let us know in the COTI Website issue forum if you find any problems.
  • We, the systems administration staff, apologize for this unexpected outage of the boards. We have resolved the root cause of the problem and there should be no further disruptions.

Sector Data File Formats (.sec)

Code:
<system mainworld="Fetters Alpha"
        hex="0101"        
        uwp="B373665-B"
        bases="A"
        zone="R"
        planetoids="2" 
        gasGiants="4"
        rockballs="4"  
        stars="2"
        allegiance="Na">
[FONT=arial,helvetica]

Quick question about bases="A" if you had a system with more than one base what would the data look like ?

[/FONT]

The attributes attached to "system" were just pulled from the UWP line; sort of a fundamental signature for the hex with no embellishment. The locations and numbers of bases have to be expansions with their own tags, probably inside their owning worlds or orbits...
 
I'd written routines into HE2 for creating an XML Sector file which contained standard UWP information usually contained in a SEC or HES file. I had plans to expand it to include System and World data as I worked on the application.

The format was very similar to the examples shown here. The primary issue was the size of the file with a standard size sector taking up upwards of 0.5mb. Whilst this is not an issue whilst dealing with UWP information, it becomes an issue when it is expanded to include System and World data, i.e. your talking about megabytes of data for a single sector.

I'll post an example of my format for comparison with the examples posted here. Peter Trevor now has the source code, so perhaps he can provide some comments on how to take this forward.

Regards,

Stuart
 
I'll start a new thread, so that I can keep track of my current preferred structure.


ArchCarrier said:
The format was very similar to the examples shown here. The primary issue was the size of the file with a standard size sector taking up upwards of 0.5mb. Whilst this is not an issue whilst dealing with UWP information, it becomes an issue when it is expanded to include System and World data, i.e. your talking about megabytes of data for a single sector.

I'd love to see your format, because I'm looking for ideas to use. You're right about file size, but I think the best approach is to call this a "data interchange" format, so it is possible to send and receive data at the system level ("please send me the data for 1910 Spinward Marches"), as well as load an entire database from a big fat file.


I would be careful with how much information you put into the file. What I mean is that whatever you put into the file you will want a standard for. For example, you have a property Government with a value 6 being a captive government. Is the table for government types the same for all versions of Traveller? If we put expected data into the files, then that data would need to be available for people (application developers) to know what acceptable values for that field is. It might be necessary to have a XML file with the valid data for the fields to be available with the standard template.

The standard is already published, by Marc, so I don't think we need to worry about this. A copy of the tables are here:

http://www.travellermap.com/formats.htm
 
Last edited:
I would be careful with how much information you put into the file. What I mean is that whatever you put into the file you will want a standard for. For example, you have a property Government with a value 6 being a captive government. Is the table for government types the same for all versions of Traveller? If we put expected data into the files, then that data would need to be available for people (application developers) to know what acceptable values for that field is. It might be necessary to have a XML file with the valid data for the fields to be available with the standard template. I do not know if this would be acceptable with Mark Miller.

Well, I think that the code is the important thing here, and you make a good point. In Traveller, specifically with the alien modules, the government codes are different between races/cultures. So the government code attribute should be standard (0-F)* but the inner text of the government XML element can be whatever is created by the developer for the more detailed extraction.

I do not know how much difference there are between versions of Traveller for system data. I think if you do multiple sections off the root node you can achieve a flexible system. Have your first section be your base that is common amongst all versions of Traveller and what is in the current files. Then you can have sections that are for version specific data. This way any tool that is reading the file can read the base section and any version specific section needed.

Code:
<TravellerRootNode>
     <CommonSectorData>
          Stuff…..
     </CommonSectorData>
     <CTSectorData>
          Stuff…..
     </CTSectorData>
</TravllerRootNode>
This will allow growth and applications can be set to read data for a specific version of Traveller. Another advantage of this (XML not my specific example) is that people can add to the standard without breaking existing applications (new applications developed to the XML file type). This is as long as they only add fields not remove or change.

THere aren't terribly many differences between versions - the biggest divergence is GURPS, I think, where the code values are different. Most versions just add detail outside the UWP OR have different values for certain codes (such as CT/MT to certain TNE worlds, IIRC). But all but GURPS use the hexidecimal values.

But other sections outside of the <worlds/> group could be useful for metadata, like trade routes and trade route links outside the sector, although that may be something better left for subsector metadata.

[EDIT]* - I went back and checked the alien modules for this. Alien government values go into the G+ range and are specific to the race, although they sometimes use the human standard.
 
Last edited:
THere aren't terribly many differences between versions - the biggest divergence is GURPS, I think, where the code values are different. Most versions just add detail outside the UWP OR have different values for certain codes (such as CT/MT to certain TNE worlds, IIRC). But all but GURPS use the hexidecimal values.

Era data is important. I.E. "here's the TNE UWP data". How do we deal with that without making the underlying structure bulky?

Ah. we need a "milieu" attribute right smack dab inside the "system" node.
 
Last edited:
The primary issue was the size of the file with a standard size sector taking up upwards of 0.5mb. Whilst this is not an issue whilst dealing with UWP information, it becomes an issue when it is expanded to include System and World data, i.e. your talking about megabytes of data for a single sector.

It may be a good idea to have a sector consist of multiple XML files. One main file to describe the typical sec data, plus meta data, route and border info, but have the detailed system and world data in separate XML files, referenced from the main file.

It makes the main file less clogged and easier to read/follow, but still allows for detailed information if an application needs it.
 
Era data is important. I.E. "here's the TNE UWP data". How do we deal with that without making the underlying structure bulky?

Ah. we need a "milieu" attribute right smack dab inside the "system" node.

So are you talking about this?

Code:
<system mainworld="Fetters Alpha"
        hex="0101"        
        uwp="B373665-B"
        bases="A"
        zone="R"
        planetoids="2" 
        gasGiants="4"
        rockballs="4"  
        stars="2"
        allegiance="Na"
 milieu="1105">

And then have whole separate system nodes for each milieu? Just trying to wrap my head around this.

Or would it make more sense to have separate UWP nodes in the worlds node that reference the milieu so there is not a duplication of other system information (like other worlds in system, which may or may not have alternate milieu data) and stellar data.
 
So are you talking about this?

Code:
<system mainworld="Fetters Alpha"
        hex="0101"        
        uwp="B373665-B"
        bases="A"
        zone="R"
        planetoids="2" 
        gasGiants="4"
        rockballs="4"  
        stars="2"
        allegiance="Na"
 milieu="1105">

And then have whole separate system nodes for each milieu? Just trying to wrap my head around this.

Or would it make more sense to have separate UWP nodes in the worlds node that reference the milieu so there is not a duplication of other system information (like other worlds in system, which may or may not have alternate milieu data) and stellar data.

I initially put it up in the UWP node, and then realized that the Bases would also need milieu information...

...and perhaps the trade routes...

...and potentially the allegiance field...

...and the trade codes...

In other words, either a large chunk of data gets pulled into the UWP, or else "social data" gets split off from "hard physical data", thereby breaking up the UWP into pieces...

...but then even "hard physical data" changes (for example, during the Grandfather milieu I expect to see fewer planetoid belts in a 1- or 2- sector radius from the Spinward Marches!).

So...

As long as this is a Data Exchange format, then (presumably) some program is serving milieu-appropriate system data to another program. Internally, I hope one would be able to store this data (and resolve milieu deltas) as efficiently as possible.
 
Last edited:
I initially put it up in the UWP node, and then realized that the Bases would also need milieu information...

...and perhaps the trade routes...

...and potentially the allegiance field...

...and the trade codes...

I had the same thought.

In other words, either a large chunk of data gets pulled into the UWP, or else "social data" gets split off from "hard physical data", thereby breaking up the UWP into pieces...

...but then even "hard physical data" changes (for example, during the Grandfather milieu I expect to see fewer planetoid belts in a 1- or 2- sector radius from the Spinward Marches!).

Splitting the UWP into those sections torpedos the file's organization, IMO, into something less than optimal.

As long as this is a Data Exchange format, then (presumably) some program is serving milieu-appropriate system data to another program. Internally, I hope one would be able to store this data (and resolve milieu deltas) as efficiently as possible.

My current thinking is that it may be more appropriate to have milieu sector data in separate files. Some instances don't even use OTU or variants, so any milieu designation doesn't even apply. And there may be some scenarios where milieu data may exist for some worlds but not others.

Mixing milieu data in the same file may cause more problems - it essentially requires that it be used when parsing data lest a mix of milieu data come out, which could also cause problems when generating an output if five out of seven worlds have year 700 data but the other two don't have any value for the attribute.

So milieu data should be an optional attribute; if the developer wants to consume it, it's there, but I don't think it needs to be there.
 
I stole the format wholeheartedly from J. Greely's MSEC format. It would need to be well documented, and the way it deals with out-sector borders is a hack. A gloriously beautiful hack that might be worth preserving for all time, but nonetheless a hack, so it bears consideration before enshrining.

What exactly is that? I'm looking at it over on Rob's other thread, and I see a list of hexes, but how is the border handled - that is, how does the renderer know how the border line of the hex is drawn around the hex itself?
 
What exactly is that? I'm looking at it over on Rob's other thread, and I see a list of hexes, but how is the border handled - that is, how does the renderer know how the border line of the hex is drawn around the hex itself?

It's clever -- I remember Greely discussing it on a thread here a couple years back. But I can't remember offhand. I seem to recall that it uses the data to figure out "inside" from "outside". Then it knows where to draw the border.

Ah, found it in his perl code. As it turns out, the data is organized carefully so that the first pair of data establish the outside border, and the logic walks the list from there. The border is on the outside of a clockwise walk. Thus if you want to exclude a region, you simply walk it counterclockwise. Greely's code also assumes that you're starting from the lowest possible column + row.

Upon thought, it seems like code could be written which only assumes a clockwise walk: hexes to the "right" are "inside" and so the border goes on the outer "left" edges.

# How regions work: The Gory Details
#
# Filled and stroked regions are exactly the same thing, a closed
# path that runs along the edge of a bunch of hexes. The only
# differences are what you set your styles to and whether you call
# $pdf->fill or $pdf->stroke at the end.
#
# With that out of the way, how do we know where to draw the path
# to make it continuous and closed?
#
# First, tell the user *precisely* how to describe a region:
# Draw a line through each hex inside the region that is along the
# outer border, starting at the lowest-numbered hex in the
# lowest-numbered column and going clockwise around the region until
# you're back at the start, recording the hex numbers as you go,
# including duplicates. The list will always end with the starting
# hex.
#
# This procedure gives us a guaranteed starting place for the path.
# By definition, the hexes to the Lower Left (0), Upper Left (1), and
# Above (2) the starting hex cannot be inside the region, so we can
# always draw along those edges, in that order. The next border hex
# must be one of Upper Right (3), Lower Right (4), or Under (5).
#
# If the next border hex is Upper Right (3), then from its point of
# view, the hex in Lower Left (0) is guaranteed to be inside, and
# it should begin checking at Upper Left (1), and draw around clockwise
# until it finds another border hex.
#
# This is a lot easier to follow with pictures, so I'll have to draw
# one and distribute it with the script. Users should be able to get
# by with the text instructions, fortunately. :-)
#
# The "neighbor numbers" work out nicely, such that to your neighbor
# in N, you are the neighbor in (N + 3) % 6, and he should test his
# neighbors starting with (N + 4) % 6, in increasing order until he
# hits his first neighbor (which will always be the next hex on the
# list).
#
# For the starting hex only, we have to keep a bit of extra state,
# to keep the path from overlapping itself. If we're not visiting
# it for the first time, we will stop testing neighbors at (5), no
# matter where we start, because (0) has already been drawn.
 
Last edited:
Code:
<system mainworld="Fetters Alpha"
        hex="0101"        
        uwp="B373665-B"
        bases="A"
        zone="R"
        planetoids="2" 
        gasGiants="4"
        rockballs="4"  
        stars="2"
        allegiance="Na">
[FONT=arial,helvetica]

Quick question about bases="A" if you had a system with more than one base what would the data look like ?

bases="S N" or "S,N" or something else ?

Multiple data items are stored in a space delimited list (ala classNames in CSS) or comma separated value list, etc.

>
[/FONT]

I would say this is your call. I am more in favor of punctuation for delimited text, but tabs are quite common also.
 
Hi there,

As far as the milieu goes. You could make that part of the document type. have a document for each milieu. I don't think you would need to read in more than one at a time.
 
I'll post an example of my format for comparison with the examples posted here. Peter Trevor now has the source code, so perhaps he can provide some comments on how to take this forward.

Actually, what I am most interested in doing is leveraging the MS SQL Server database on my website as a central exchange repository, connecting to it via web services and swapping ADO.NET datasets.

But if a genuine new format arises out of this thread (one that is going to be used in other tools) then I want to code towards that format as well.
 
What exactly is that? I'm looking at it over on Rob's other thread, and I see a list of hexes, but how is the border handled - that is, how does the renderer know how the border line of the hex is drawn around the hex itself?

I don't know about the MSEC format, but in Universe I assigned an allegiance to a hex (even if empty) rather than a system. The renderer would draw a border between any two adjacent hexes where the allegiance codes were different.

(In fact if you look at the sample database that comes with Universe you will see that I set the empty hexes for the Spinward Marches to create the correct border, but didn't bother for other sectors.)
 
Actually, what I am most interested in doing is leveraging the MS SQL Server database on my website as a central exchange repository, connecting to it via web services and swapping ADO.NET datasets.

But if a genuine new format arises out of this thread (one that is going to be used in other tools) then I want to code towards that format as well.

Conversely, if someone uses an interchange solution currently, then that's a candidate for adoption. A system used is better than a proposal unused.

In fact, these threads currently just serve as a thought-provoking exercise, which is just fine by me.
 
Conversely, if someone uses an interchange solution currently, then that's a candidate for adoption. A system used is better than a proposal unused.

In fact, these threads currently just serve as a thought-provoking exercise, which is just fine by me.

I agree with this statement. I have viewed all of this as a brainstorming session. :)
 
Back
Top