• Welcome to the new COTI server. We've moved the Citizens to a new server. Please let us know in the COTI Website issue forum if you find any problems.

MySQL development lies within!

So, after years of hearing about the game, and seeing the odd used book, I finally picked up Traveller (Mongoose in my case). And love it. I finally found myself in a local gaming group, they are willing to let me run a game, and my soon-to-be players want to try some role-playing outside of the context of swords and sorcery.

However, the idea of picking up decades' worth of history from OTU is slightly overwhelming. So, running a game IMTU it will be. MTU will be starting from scratch, sort of building it as my players explore, and tacking some of the history on later, and in intra-session prep. But a galaxy's worth of star systems (heck, a sector's worh!) is a lot of data to keep track of.

Good thing I like computers. And coding. And have been in need of an excuse to learn more complex SQL than
Code:
SELECT foo FROM bar WHERE baz = bletch;

So I have taken it upon myself to concoct myself a MySQL database which will store store UWPs, star systems, subsectors, and sectors. And while I'm at it, tackle character (and thus NPC) generation and storage. And why not- how about ships?

I've learned a metric truckload of SQL in the past few days, and have made much progress including taking discrete system data the like of:
Code:
 mysql> select * from system where id = 3;
+----+-----------+---------+-----+----------+------+------------+-------------+------------+------------+-----+------------+----------+-------------+-----------+--------------+
| id | subsector | name    | hex | starport | size | atmosphere | hydrosphere | population | government | law | tech_level | bases    | travel_code | gas_giant | culture_code |
+----+-----------+---------+-----+----------+------+------------+-------------+------------+------------+-----+------------+----------+-------------+-----------+--------------+
|  3 |      NULL | Salaxis |   0 | C        |    6 |         12 |          10 |          3 |          5 |   7 |         10 | Research | AMBER       |         1 |            0 |
+----+-----------+---------+-----+----------+------+------------+-------------+------------+------------+-----+------------+----------+-------------+-----------+--------------+
.. and making it into a UWP the like of:
Code:
mysql> call gen_uwp(3);
+---------+------+-----------+-------+-------------+-------------+
| Name    | Hex  | UWP       | Bases | Trade Codes | Travel Code |
+---------+------+-----------+-------+-------------+-------------+
| Salaxis |    0 | C6CA357-A | R     | Fl Lo Wa    | AMBER       |
+---------+------+-----------+-------+-------------+-------------+
Granted, I'm certain that the bulk of my SQL code is atrocious and ugly and probably stored in a manner that will paint me into a rebuilding-the-design-from-the-ground-up corner in a week or two.

However I'm nearly at the point where I will have an entirely SQL-based star system generator, so I thought I would come here to do two things:

Firstly, I'd like to offer to share my schemas with anyone who would be interested, and secondly, to ask if this wheel has been invented before and pick some brains on some things.

For example (and keep in mind I'm completely new to traveller), I just today actually took a look at how the hexes are numbered in a subsector or sector map. I was about to weep about the prospect of generating an arbitrary Jump-6 map for a given sctor until I finally noticed that, if the first hex in subsector A is 0101 and the last in subsector P is 3240, then the first two digits of the hex make a very nice X coordinate, and likewise the second two for Y. However, 0402 is 'southwest' of 0502, while 0302 is 'northwest' of 0402. This will be in interesting problem to solve. So too will be determining what systems lie within, say, Jump-4 of a given hex.
 
Last edited:
Could I encourage you to keep it up and keep this thread alive? My very first self-generated project was on my old Z80 based computer in BASIC doing character generation from Book 1 all those years ago. Traveller totally lends itself to projects that stretch your programming skills. In the process, you'll be teaching us all a bit of SQL.

Don't worry about code being ugly and unfinished - all decent code is written iteratively and you'll polish it as you go, in the same way as you're polishing YTU as you go. My first character generator hard-coded all of the dice rolls, and I kept adding extra services by extending the code, had no GOSUB calls (I used GOTO for everything - total spaghetti code, but heck, I knew how it worked. I eventually started figuring out separating data from code, and that was another lesson for me.

I'm now working in HTML/Javascript and PHP to get Traveller stuff done (I'm too embarrassed to post anything, you're braver than I am).

Have fun! A great start to a very happy journey.
 
Sorry, just reading your post again, if you haven't already checked it out, and sorry if I'm teaching grandma to suck eggs here: Heaven and Earth is worth a look and accompanying the programme is the entire OTU in a text-based format ("WBS").

Just on hexes: yes, the first two digits and last two digits make X and Y co-ordinates. Just to help you out, a move from any given hex of co-ordinates X and Y will translate to new co-ordinates X' and Y' in the following directions (using "north", "south" for up and down, etc): The % symbol here means the 'mod' operator - i.e. divide by the denominator but ignore the result and take the remainder only. The 'Abs' means take the absolute value.

North: X' = X, Y' = Y - 1 (yes, the obvious cases first)
South: X' = X, Y' = Y + 1
North-East: Y' = Y - X%2, X' = X + 1
South-East: Y' = Y + Abs((X%2)-1), X' = X + 1
North-West: Y' = Y - X%2, X' = X - 1
South-West: Y' = Y + Abs((X%2)-1), X' = X - 1

The assumption is that the first column is numbered 'one' and the top-left hand hex only has hexes to the south-east and south. The Mod 2 operations are an arithmetical way of expressing a test for whether the current x-co-ordinate is odd or even - so if it's even in this schema, then a move to the north-east will not change the value of Y, but will add one to the value of X.

Hope that helps a little.

Obviously in the OTU, "North" is Coreward, "West" is Spinward, etc.
 
Welcome to fun world of SQL, DopeGhoti.

If you can get hold of InterBase 6 (or possibly Firebird) then there is already a database designed and populated you can download. Check here, it is the database used by Universe. (You have to pay if you want the Universe program but the database itself is free.)

(Note to self: a lot of the links on that page need fixing. Do tonight.)
 
Welcome to fun world of SQL, DopeGhoti.

If you can get hold of InterBase 6 (or possibly Firebird) then there is already a database designed and populated you can download. Check here, it is the database used by Universe. (You have to pay if you want the Universe program but the database itself is free.)

(Note to self: a lot of the links on that page need fixing. Do tonight.)

Sadly, I've not the extra money at the moment to spend on Universe, nifty though it does look. I have seen some of the information posted on its successor-to-be, Universe II, and it definitely looks up my alley, and may have a spot on my buy-list once it's released and I have more disposable funds.

Onjo said:
Sorry, just reading your post again, if you haven't already checked it out, and sorry if I'm teaching grandma to suck eggs here: Heaven and Earth is worth a look and accompanying the programme is the entire OTU in a text-based format ("WBS").

I took a brief look at H&E, but it didn't run on my machine, and I hadn't yet had the time to install an earlier generations' Windows in a VM and try it there. Thanks for the suggestions, however- looking at H&E's data files never even occurred to me!
 
Last edited:
Could I encourage you to keep it up and keep this thread alive? My very first self-generated project was on my old Z80 based computer in BASIC doing character generation from Book 1 all those years ago. Traveller totally lends itself to projects that stretch your programming skills. In the process, you'll be teaching us all a bit of SQL.
You most certainly could. You may regret that, though, once you see my SQL code. :)

Don't worry about code being ugly and unfinished - all decent code is written iteratively and you'll polish it as you go, in the same way as you're polishing YTU as you go.
Despite many colleagues' misgivings, I've found that way to work best for me as well. And with a learning experience like this, more eyes are nearly always better than two.
My first character generator hard-coded all of the dice rolls, and [..] had no GOSUB calls (I used GOTO for everything - total spaghetti code, but heck, I knew how it worked. I eventually started figuring out separating data from code, and that was another lesson for me.
I first started coding with Integer BASIC on the Apple //. There was no GOSUB, like I ultimately found in AppleSoft BASIC. And it was like a whole new world when I started learning Pascal (you can have a subroutine in a subroutine?!) and C. Recently, I've started tackling OOP and SQL, which has been just as fun and eye-opening.

I'm now working in HTML/Javascript and PHP to get Traveller stuff done (I'm too embarrassed to post anything, you're braver than I am).
Perhaps we could combine our forces? I'm conversant with PHP and HTML, but have no sense of design (I'm more of an algorithm guy than an interface guy), but between the two of us, maybe we can slap together a pretty good generalized 'IYTU' Traveller app!

Have fun! A great start to a very happy journey.
I certainly plan to!
 
Just on hexes: yes, the first two digits and last two digits make X and Y co-ordinates. [..]a move from any given hex of co-ordinates X and Y will translate to new co-ordinates [as follows]

  • North: X' = X, Y' = Y - 1 (yes, the obvious cases first)
  • South: X' = X, Y' = Y + 1
  • North-East: Y' = Y - X%2, X' = X + 1
  • South-East: Y' = Y + Abs((X%2)-1), X' = X + 1
  • North-West: Y' = Y - X%2, X' = X - 1
  • South-West: Y' = Y + Abs((X%2)-1), X' = X - 1

At first glance, I figured that I'd be hoping for a Modulus mathematical operator in SQL. I didn't think of using absolute values. Nice.

Hope that helps a little.
It helps a lot! :)

Obviously in the OTU, "North" is Coreward, "West" is Spinward, etc.
Conventions that are easy to stick with.

If you're interested, I'll share some of what I've got thus far, code-wise.
 
So, I've done a little redesign of my tables (as I was rather expecting I would have to). For simple things, my MySQL SET datatype (sort of the bastard offspring of the BIT (or bitfield) datatype and the ENUM datatype is really neat and helpful as long as you don't have to cross-reference or want to make your database more expandable.

I've also learned about link tables for many-to-many relationships. For those who are interested, I have attached the logical structure of my DB as is exists now.

I'll need to re-enter the trade code generation data, as that too has completely transformed. I'll post the before-and-after of my trade code generation procedure once the new one is written so you can see how much simpler the code is to write when you refactor the schema as I have.

Most of the tables have been collapsed so as to just show their names, but I've left the system table expanded so you can get an idea of how it's put together.

Click for bigger view:

IMTU SQL Structure

(I told you I'd keep y'all posted!)
 
Now, for my first dumb question, would I be right in thinking that you're going with a many-to-many relationship for each characteristic so you've got a table that explains the value of each (e.g. Atmosphere = 6 can be looked up as "Standard atmosphere" etc.)?

Secondly, I'm interested by "Atmosphere has gear" - what's that all about? I'm also interested in "Starport Facility". Bear in mind, I'm still mucking around in MegaTraveller / Classic Traveller so these might be Mongoose things.

I tried an approach to world generation / data reading using PHP and XML - happy to share it. Basically, instead of treating each world as a record, each world is an instance of a class.
 
Now, for my first dumb question
No such thing, friend!
would I be right in thinking that you're going with a many-to-many relationship for each characteristic so you've got a table that explains the value of each (e.g. Atmosphere = 6 can be looked up as "Standard atmosphere" etc.)?
More or less. Most of them are one-to one (each system has one atmosphere, one starport, etc.). As for the look-ups:
Code:
mysql> select * from atmosphere;
+------------+--------------------+--------------+--------------+
| atmosphere | description        | pressure_min | pressure_max |
+------------+--------------------+--------------+--------------+
|          0 | None               |        0.000 |        0.000 |
|          1 | Trace              |        0.001 |        0.090 |
|          2 | Very thin, Tainted |        0.100 |        0.420 |
|          3 | Very Thin          |        0.100 |        0.420 |
|          4 | Thin, Tainted      |        0.430 |        0.700 |
|          5 | Thin               |        0.430 |        0.700 |
|          6 | Standard           |        0.710 |        1.490 |
|          7 | Standard, Tainted  |        0.710 |        1.490 |
|          8 | Dense              |        1.500 |        2.490 |
|          9 | Dense, Tainted     |        1.500 |        2.490 |
|         10 | Exotic             |         NULL |         NULL |
|         11 | Corrosive          |         NULL |         NULL |
|         12 | Insidious          |         NULL |         NULL |
|         13 | Dense, High        |        2.500 |         NULL |
|         14 | Thin, Low          |         NULL |        0.500 |
|         15 | Unusual            |         NULL |         NULL |
+------------+--------------------+--------------+--------------+
16 rows in set (0.00 sec)

Secondly, I'm interested by "Atmosphere has gear" - what's that all about?
I haven't populated those tables yet, but different atmospheres need different sets of equuipment for survival (some need a Vacc Suit; some need just an air supply, some need an air filter; some need an air filter and a respirator. The linking table will make it easy for me to query for a list of what gear is needed. (respirator and filter for 2; just a filter for 7, for example).
I'm also interested in "Starport Facility". Bear in mind, I'm still mucking around in MegaTraveller / Classic Traveller so these might be Mongoose things.
It's probably a combination of me and Mongoose mucking about with traditional terminology. A starport can have some number of: Limited repair, Repair, and three classes of Shipyard (small craft, spacecraft, and capital shipyards). It's another of those many-to-many relationships, supplemented by the starport table itslef:
Code:
mysql> describe starport;
+-----------+------------------------------------+------+-----+---------+-------+
| Field     | Type                               | Null | Key | Default | Extra |
+-----------+------------------------------------+------+-----+---------+-------+
| starport  | enum('X','A','B','C','D','E','F')  | NO   | PRI | X       |       |
| quality   | varchar(15)                        | NO   |     |         |       |
| berth_fee | int(5)                             | NO   |     | 0       |       |
| fuel      | enum('NONE','UNREFINED','REFINED') | NO   |     | NONE    |       |
+-----------+------------------------------------+------+-----+---------+-------+
I tried an approach to world generation / data reading using PHP and XML - happy to share it. Basically, instead of treating each world as a record, each world is an instance of a class.
If you're willing to share, I'd love to see it. Munching on code is a lot healthier than potato chips! :)

Once I have the many-to-many tables populated, they'll make a lot more sense.
 
You can download a ZIP file of the code here: Link. If you've got PHP set up on your machine, then after you've dumped the contents of that ZIP file in your wwwroot directly or equivalent, run worldGen.php from your localhost for a demonstration. It generates a sector for you with Vilani world names.

The idea is that the file stdWorldGen.xml can have other formulae / values loaded into it for a set of arbitrarily named characteristics for generation, and the mainWorldClass can convert any world it generates or loads from a text file into a XML node - i.e. can construct a data file of your sector. What I'd love to do is get it working with SQL because of (a) speed, (b) speed, and not to mention (c) speed.

Some of the classes might seem a bit obsessive, but a lot of it is really an exercise in teaching myself PHP and the kinds of data structures I want on a project I'm working on. Being familiar with Traveller gives me a clear objective.

Enjoy!

P.S. Sorry, the code is buggy, I haven't got bases generating correctly, and I can see trade classification mistakes. But hopefully the overall approach gives you something to go on.
 
Last edited:
Some of the classes might seem a bit obsessive, but a lot of it is really an exercise in teaching myself PHP and the kinds of data structures I want on a project I'm working on. Being familiar with Traveller gives me a clear objective. Enjoy!
I'll definitely take a look at it; it sounds like you used Traveller as a way to learn PHP in the same way that I'm using it to learn SQL- great minds think alike, no?

I just finished populating one of the linking tables, so now's a good time to show you what I mean:

Let's say I wanted to know what all the trade codes were that involved having no atmosphere:

Code:
mysql> select type, trade_code from trade_code natural join code_has_class where attribute='atmosphere' and uwp_class = 0;
+------------------+------------+
| type             | trade_code |
+------------------+------------+
| Asteroid         | As         |
| Ice Capped       | IC         |
| Industrial       | In         |
| Non-Agricultural | Na         |
| Vacuum           | Va         |
+------------------+------------+
5 rows in set (0.00 sec)

As a small example of how the many-to-many linking table itself looks, here are all of the records for Asteroid and Vacuum worlds:
Code:
mysql> select * from code_has_class where trade_code in ('As', 'Va');
+------------+-----------+-------------+
| trade_code | uwp_class | attribute   |
+------------+-----------+-------------+
| As         |         0 | SIZE        |
| As         |         0 | ATMOSPHERE  |
| As         |         0 | HYDROSPHERE |
| Va         |         0 | ATMOSPHERE  |
+------------+-----------+-------------+
4 rows in set (0.00 sec)

The procedural upshot of this is, in the stored procedures I'm about to be rewriting, it'll be a lot easier to step over the results of a SELECT with a CURSOR than it was to have a huge string of IF THIS = THAT THEN SET foo = concat( foo, 'Ba'); statements.
 
not sure if MySQL has dynamic stored procedures, but that's what I used to do for making search queries. In general, and this won't apply to small scale stuff, you don't want to use cursors (even if you have all 11,000 worlds in your DB, that is really small scale still)

For dynamic SQL, the procedure variables are defined as null on the way in as a default value, and if not null (i.e., you use the variable when calling the procedure) then you add that to your SQL statement. Then, when done, you do the exec sql with the parameters generated. That way you have one statement that covers a wide range of query selection. A bit of a pain to set up, but a LOT easier to understand than looping via cursors, so you have a higher level of maintainability.

something like:

CREATE PROCEDURE proc AS
@variable1 int AS NULL,
@variable2 varchar(255) AS NULL

define @sqlStatement nvarchar(2048)
define @paramList nvarchar(2048)

SET @sqlStatement AS 'SELECT * FROM TABLE WHERE 1=1'

IF @variable1 IS NOT NULL
SET @sqlStatement AS @sqlStatement + ' AND table.var1 = ' + @variable1

(I think - it's been almost a year since I last used SQL Server in a development position, but that's the general idea)

Of course, some people also don't recommend dynamic SQL - as always, YMMV!
 
Success!

I've finally gotten away from hard-coded lists of table fields in my UWP generator, and can still get from this:
Code:
mysql> select * from system where system = 3; select * from system_has_base where system = 3; select * from code_has_class where trade_code in ('Fl', 'Wa', 'Lo') order by trade_code, attribute, uwp_class asc;
+--------+---------+------+----------+------+------------+-------------+------------+------------+-----+------------+-------+-------------+------------+-------+-----------+---------+-------------+-------+
| system | sysname | hex  | starport | size | atmosphere | hydrosphere | population | government | law | tech_level | bases | travel_code | pop_prefix | belts | gas_giant | culture | temperature | notes |
+--------+---------+------+----------+------+------------+-------------+------------+------------+-----+------------+-------+-------------+------------+-------+-----------+---------+-------------+-------+
|      3 | Solaxis | 0000 | C        |    6 |         12 |          10 |          3 |          5 |   7 |         10 | R     | AMBER       |          0 |     0 |         1 |      15 |           3 | NULL  | 
+--------+---------+------+----------+------+------------+-------------+------------+------------+-----+------------+-------+-------------+------------+-------+-----------+---------+-------------+-------+
1 row in set (0.00 sec)

+--------+------+
| system | base |
+--------+------+
|      3 | R    | 
+--------+------+
1 row in set (0.00 sec)

+------------+-----------+-------------+
| trade_code | uwp_class | attribute   |
+------------+-----------+-------------+
| Fl         |        10 | ATMOSPHERE  | 
| Fl         |        11 | ATMOSPHERE  | 
| Fl         |        12 | ATMOSPHERE  | 
| Fl         |        13 | ATMOSPHERE  | 
| Fl         |        14 | ATMOSPHERE  | 
| Fl         |        15 | ATMOSPHERE  | 
| Fl         |         1 | HYDROSPHERE | 
| Fl         |         2 | HYDROSPHERE | 
| Fl         |         3 | HYDROSPHERE | 
| Fl         |         4 | HYDROSPHERE | 
| Fl         |         5 | HYDROSPHERE | 
| Fl         |         6 | HYDROSPHERE | 
| Fl         |         7 | HYDROSPHERE | 
| Fl         |         8 | HYDROSPHERE | 
| Fl         |         9 | HYDROSPHERE | 
| Fl         |        10 | HYDROSPHERE | 
| Fl         |        11 | HYDROSPHERE | 
| Fl         |        12 | HYDROSPHERE | 
| Fl         |        13 | HYDROSPHERE | 
| Fl         |        14 | HYDROSPHERE | 
| Fl         |        15 | HYDROSPHERE | 
| Lo         |         1 | POPULATION  | 
| Lo         |         2 | POPULATION  | 
| Lo         |         3 | POPULATION  | 
| wa         |        10 | HYDROSPHERE | 
+------------+-----------+-------------+
25 rows in set (0.00 sec)
to this:
Code:
mysql> call show_uwp(3);
+---------+------+-----------+-------+------------+-------------+
| System  | Hex  | UWP       | Bases | Trade Code | Travel Code |
+---------+------+-----------+-------+------------+-------------+
| Solaxis | 0000 | C6CA357-A |       | Fl Lo Wa   | AMBER       | 
+---------+------+-----------+-------+------------+-------------+
1 row in set (0.77 sec)
There has been much learning about stored procedures, user variables, and prepared statements. The code is still ugly, but at least now it's procedural. Here's the segment of show_uwp that generates the Trade Codes:
Code:
    create temporary table valid (
        v int(2)
    ); 

    -- Generate trade code column
    open tcode;
    code_loop: LOOP
        set @applies := 1;
        fetch tcode into tmpcode;
        if ( done1 = 1 ) then
            close tcode;
            leave code_loop;
        end if;
        open attr;
        attr_loop: LOOP
            fetch attr into tmpattr;
            if ( done1 = 1 ) then
                close attr;
                set done1 := 0;
                leave attr_loop;
            end if;
            -- Okay, we have a trade code and an attribute to test!
            truncate valid;
            set @tc := concat( '\'', tmpcode, '\'' );
            set @ta := concat( '\'', tmpattr, '\'' );
            set @qs := concat(
                'insert into valid select uwp_class from code_has_class where trade_code=',
                @tc,
                ' and attribute = ',
                @ta);
            prepare dq from @qs;
            execute dq;
            deallocate prepare dq;
            set @qs :=  ( concat(
              'select ', 
             tmpattr, 
             ' from system where system = ',
              sys_id,
              ' into @av') );
            prepare dq from @qs;
            execute dq;
            deallocate prepare dq;
            select @av in ( select * from valid ) into @flag;
            if @flag = 0 then
              set @applies := 0;
            end if;
        end loop attr_loop;
        if not (@applies = 0) then
          set t_tradecode := concat( t_tradecode, tmpcode, ' ' );
        end if;
    end loop code_loop;
    drop temporary table valid;
Clearly, I figured out the joy that is undeclared variables half-way though writing this code; I'll be getting rid of a lot of DECLARE statements shortly. Now that that's tackled, procedurally generating the Bases column will be cake.
 
Impressive stuff! ... I've had MySQL set up on my machine with PHP for a while. Mind if I copy & paste your code? Just looking at the beginning and end results, I'm impressed. I want to muck around with it a bit if that's OK.
 
Impressive stuff! ... I've had MySQL set up on my machine with PHP for a while. Mind if I copy & paste your code? Just looking at the beginning and end results, I'm impressed. I want to muck around with it a bit if that's OK.

Sure thing- bear in mind that's only part of the stored procedure. When I have a free moment at work, I'll post the entire schema so far, and the entire procedure. As you muck about, if you think of anything that would be wise to add, or make any improvements, kindly share alike, eh? :)
 
Well, I would attach the .sql file if I could attach files to messages- is that not enabled on this board?

Zip it and upload it to the file library, not the general discussion. If you then ask me nice, I'll edit a link to the file in...
 
Back
Top