Metagame: Logs, API Hooks, and You

[Aesir Rising]

Sometimes a game makes me want to stop playing to take notes. I'm not alone, as we see here:

Evochron Mercenary Universe Map (SPOILER ALERT)

The act of taking notes is inconvenient when you're otherwise just trying to play a game. And then I get hung up on what notes to take, how I want to format things, and how best to store, and later search, query, and otherwise retrieve that data.

Omitting a bit of data at the start of a note-taking project is risky, as it not always fun or practical to return to prior points in a game to recapture the missing info.

Then there is the medium used for note-taking.

Sometimes I use Notepad. Sometimes I use a notepad. Sometimes I use an in-game notes feature. Sometimes I use a spreadsheet or database. Sometimes I use screenshots. Et cetera, ad nauseam.

Usually I just give up before ever realizing completion of a data-collection project, having failed to settle on an accurate method of storage and retrieval, and finding the inconvenience of taking notes (including task-switching to do so) a problem that never goes away.

This game has a note-taking feature. But it is just a simple in-game scratch pad, with no real way to formalize either data storage or retrieval, and no mechanism that helps avoid re-typing essentially what it is that you're already seeing on screen.

Over time, I'd like to see this change. Over time, I'd like to see some real data collection aids for this game.

The first change I'd like to see is to provide logging features geared towards a player playing the game (not logging for diagnostics or troubleshooting - I'm sure that exists already).

These logs would record a snapshot of data otherwise visible to the player (e.g., information on the F1, F3, and F4 displays).

There are many ways to go about this, and many potential data points to collect, but I'd like to start somewhere and build up as we go. This approach allows us to tweak the specific implementation, and later - over time- to add more data points collected by the logs.

Assuming a logging implementaton gets realized, here are some user requirements I'd foresee. These aren't listed in any order, so some features could come later than others.

• Log file in delimited text. Alternately, XML, but I feel CSV or tab-delimited is more accessible to a larger number of people (which is important, in order to maximize the impact of the development dollars spent on implementing the feature).
• In-game hotkey(s) to take log snapshots. Fully-implemented in-game window (like what we see for wing formation commands) for all data logging functions.
• Trade data: Log a snapshot of pricing for everything bought/sold, including data/time, pilot name, station/location, and source (station price, ship-to-ship trade offers), and an indication as to whether faction standing affected the pricing (I don't know if it does, but if it does, then the faction standing that was in affect at the time of the snapshot is useful).
• Exploration data: Track entry/exit date/time of each square on the nav map for a pilot, and the coordinate representation of those squares. Track entry/exit date/time of stations, hanger docking, planetary stations, planet atmosphere entry/exit, and other data useful to answer any questions about "who has been where, and when?"
• Combat logging: Record shots fired and those that hit, the target that was hit, along with location/date/time.

That's enough from me for now. The underlying objective is to automate data recording for most or all of what we can otherwise see in-game already. The scope, imo, should not include data a pilot or player couldn't normally see on screen while playing the game.

If I had a preference, I'd start with recording exploration data (F1) and market data (F3 and maybe F4).

Players can then look into making spreadsheet templates, local and online databases, to store then query and report on the collected data.


[Edited on 10-13-2010 by Aesir Rising]

[Aures]

Good idea, don't forget navigation logs should include the system designation of any sector you enter. I don't care that much about date and time information, but being able to automatically record system designations would be invaluable for me.

I currently have a spreadsheet with a couple of thousand entries of just the sector coordinates plus system designation. Filling it in occupies the time between jumps nicely, but it would certainly be easier to just focus on jumping and let the computer handle the logging.

Goods price information would be really helpful. It is quite a pain to take proper note of it at the moment, especially with the random factor thrown in. If you could just automatically log the price of all available goods (and everything on your ship that you could sell but is not available to buy) on docking it would be a big help.

[Marvin]

Originally posted by Aesir Rising

Sometimes I use Notepad. Sometimes I use a notepad. Sometimes I use an in-game notes feature. Sometimes I use a spreadsheet or database. Sometimes I use screenshots. Et cetera, ad nauseam.

Sounds like somebody needs to get organized.

[Rush]

Well a nav-map that makes you see anything in the galaxy you have already seen would be very useful.
Just like SeeJay's map, but automatically updated while exploring. And the possibility to see on the nav-panel not only the sectors around you, but any point you have already explored...

[Aesir Rising]

I should make it clear that as a proof of concept, and also to avoid concerns about exploits or otherwise making the game too easy, I'd like a snapshot of data to occur in most cases only when a player has initiated an action or input. For example, when you press a button as described by the following implementation and use-case:

• If I click the News button while docked, I'd like to have a row written out with the following:
• pilot name, location, date, time, commodity/item (a column for each commodity name), price ( a column for each commodity price)

I know it is technically possible to just automagically track all prices for all systems for all times every time the market changes. I don't want that though.

Likewise, for exploration data, I'd like a row written out when the player presses F1 (whatever is bound to the nav console).
This includes pilot name, date, time**, location type, location name, game universe coordinates and ship navigation/jump drive coordinates if this differs from whatever the game uses internally to track location (this is important for 3rd party mapping utilities, like Seejay's site).

** Aures: It isn't immediately clear why date/time is useful for nav information, but I'll try to explain:
It can allow players to query not just how many times they've travelled to/from a location, but *if* they've travelled to/from a location. This is the field I would use as a basis for a map or player query that displays which locations have been explored (if a location has not date/time, that pilot has not been there yet).

Finally, I am purposely vague in using the term "location". This is because I am ignorant about what object types exist and don't want to leave anything out. The goal is to capture information though, so generally, if something displays on the map or even just through the cockpit window, I want it logged.


Oh, and if I had to really pick just one thing to start with, I suppose market data snapshot whenever a player hits the News button at a station. This is because it is easiest to implement and record. I confess I would completely have a geekgasm if I could import a bunch of CSV records into OpenOffice and create market trend graphs and pie charts and things. Later on, if the idea gains traction, I'd like to add records for crewmember hires and salary, ship-to-ship trade requests, and any other buy and sell transactions a pilot makes.

Originally posted by Marvin
Sounds like somebody needs to get organized.

Yes, that is what I'm trying to do.

[Edited on 10-13-2010 by Aesir Rising]

[Edited on 10-13-2010 by Aesir Rising]

[Marvin]

I was wondering what kind of affect all this stuff has on multiplayer. Granted, the executable isn't getting any bigger, but all this additional scanning of keycodes might be stressing out some gamer's CPU. Like mine. 'Cause it seems, with every add-on to the original game, I see more and more lag in MP ... even though the number of players has declined. Not a game-breaker, by any means ... but disturbing nonetheless.

[Aures]

I wasn't suggesting that nav date/time data shouldn't be captured, just that you hadn't mentioned system designations and those are what I would really want to capture. I am not saying anything you are suggesting shouldn't be done, just that there are others things you did not mention that would be really useful.

I am not sure about your implied structure where all locations are present but only marked with a date/time once visited. The sector range is BIG (I read the exact figures for Legends once, I assume Mercenary is the same or bigger). It includes billions of sectors, most of which you will never visit. Having a list of all of them that gets info added to it would require a lot of wasted storage space.

I am not sure about requiring player input to capture information. I would quite like to capture information when I leave the game running by itself while the autopilot takes me a long distance. Having to press a button between each jump kind of defeats the purpose. If it could be configured to be off, manual or automatic that would be fine.

If it could also automatically log the location of any objects you come across that would be really useful too. But, that might not be desirable as it would allow a brute force approach to exploration where you just check the game every couple of hours to refuel and set a new autopilot destination. I think that would be fine for compiling a territory map like me, but I am not sure about finding actual objects that way.

[Aesir Rising]

Logging impact on MP performance? Hmm. Okay...

We need to know what is getting logged.

• Is it data available locally? (Part of stuff installed when you install the game)
• Is it data that sits on the server only? (Something in the dedicated server that isn't on your PC)
• Is it data that is transfered to/from the server? (Something that is calculated on the server, then sent to the players)?

Navigation information would likely be already on your PC. You've already got the whole universe sitting in that *.evo file.

The impact on MP for the type of data I mentioned for navigation and exploration logging then is based on whether your PC can handle the logging load or not. But that isn't an MP concern, that is a concern for SP as well if your PC cannot handle some given load. And if it cannot do so, then don't enable logging on your PC.

Market information would likely be on the server-side in an MP game - and sent to any player requesting the information. This is what happens when you press F3\\News right now this moment. What is the impact of this on MP now? Is it bad enough that you'd ask for MP servers to not track local system pricing for commodities and other trades? I doubt that idea would gain much steam. Providing the ability to log this data just writes that data to your local PC, and isn't something that then needs to get shared out to other players since they're already getting the same data now directly from the server.

There is however a set of data that I would find interesting that sits solely on an MP server, and must be transmitted to all MP players, back and forth.

Combat logging.

And I'd put that idea on hold until the MP impact can be assessed. And if it is ultimately determined that MP cannot handle combat logging, then I'd ask that it be implemented anyway, so that single player games can access the information.

You mentioned a concern about "..additional scanning of keycodes.." and I have to ask if the MP server is lagging because players are pressing buttons and keys now. If it is, there are some big issues I can't address in this post. But part of the diagnosis of *why* MP lags when people press a lot of buttons would, ironically, involve detailed logs of player activity.

[Marvin]

Sorry, the question was directed at Vice. And scanning means a programming loop that continuously checks for input. The more sources it checks, the longer it takes to complete each cycle of the loop. There are actual programming terms for this but I'm having another Yoda moment.

Only Vice would know for sure. And only Vice an answer your "need to know" suppositions.

[Daedalao]

I use a green book. Anyone previously in the US military should know what that is. It has coordinates and headings for all kinds of interesting things. It's my rudder if you will. It also has contact information which is being compiled into a sort of... risk assessment for certain sectors. I also have my flight time through most sectors written down. But again, it's pen to paper mainly, with some data being on my computer after it's been recorded.

[Aesir Rising]

Aures: "System Designation"? That falls under, "....and other data useful to answer any questions about "who has been where, and when?"

I wasn't looking to define the log data structure at this point. This is the part where I float the idea to see if it sticks. If it does gain some steam, then yeah, let's get specific about data structure for the logs, so we don't implement something useless. Any field names I did mention were to provide some examples and context for the request, in order to flesh the idea out for those interested in understanding the request.

To your other point... why would you need to store a list of billions of unknown location from the start? That does conflict with what I'm asking for - I very specifically don't want spoilers. I only want stuff logged that I would otherwise have seen in the game. But regardless, you wouldn't need to start off with some monolithic list of all known points in the "universe" because you've already got that when you installed the game (at least, in the form of a set of instructions used to create the game universe). If you see something wrong, please let me know. Maybe the idea doesn't work as I'd described it?

Marvin, of course - how could I have jumped in like that? I doubt anyone other than Vice can respond to your question about the thing that is not scanning keycodes, and how it impacts multiplayer performance. Maybe you could start your own thread on the subject to avoid getting off-topic responses to your questions from me.

[Marvin]

NP with your response. I was just pointing out that it's an observation about MP that only Vice could clarify. And the reason I posted it here is because this is the latest in a rather lengthy series of requests for "stuff" that's secondary to the game. Nice to have but, like $200 tennis shoes and 4' flat-screen TVs, are not always affordable. Meaning, it's getting to the point where my CPU and internal memory seem to be about maxed out.

[Ravenfeeder]

I do a wee bit of logging: pen, paper, and my wee bit of brain!

[Aures]

Originally posted by Aesir Rising
(if a location has not date/time, that pilot has not been there yet).

Sorry, I must have misunderstood what this meant. As long you aren't suggesting a predefined location list that gets a date/time put next to it when first visited by the pilot then no problem.

As to the sector designation, yeah that falls under the heading of "who has been where, and when?", sorry if I got ahead of the discussion by stressing an important specific rather than just voicing my support for the concept.

At this stage I'd rather focus on how automated the logging features would be since you bring it up. There is a big difference between a button that is basically a print screen that parses the information into a file for you and an automated logging function. I agree that capturing everything all the time being the only way to use it would not be good. But, unless there is a reason to not let a specific piece of information be captured automatically it should be an option. For example, if I want to capture a snapshot of prices every time I dock it is much better to set the option once than to click an extra button every time I dock. Of course, I should also have the option to only capture the information when I press a button.

[Aesir Rising]

Aures:
I personally would like to have market data collected every time I docked. Just like I'd like every trade offer (and transaction) logged, and every hire and crew wage payment, and fleet hire, and docking fee, mission reward, and every other payable/receivable I don't know about logged without my intervention (other than enabling the logging option).

However, not everyone likes the idea of logging and automation in general, so I was trying to err on the side of caution in making the request. Since when talking about automation I don't want the general idea of logging to get mired into an argument about whether or not it makes the game too easy, or whether such automation goes against the "whole philosophy" of the game. (I don't really know what that means either, but it was one of the two main arguments I had to contend with when my idea to auto-save when docking got shot down. And I don't want logging and data APIs for 3rd party utilities to get shot down).

Anyway, on the topic of navigation logging and general exploration automation. I don't suppose I could ask you to 'trust me' and hold off on getting too deep into that? I suppose not. Ok, the idea isn't fully-fleshed out in my mind with respect to specific and complete implementation details, but the situation you describe - where you're run your ship on auto-pilot for hours to get some exploring done? That game play feature sucks.

But we don't have an alternative approach that nets the same or better results right now. However, there is a conceptual ideaI've been chewing on for a bit that I think is simultaneously more fun and serves as a money-sink, especially for late and end-game pilots that have all the gear they already need and are really focused on exploring the far reaches.

This as-yet-unmentioned idea has little to do with logging per se, though logging would likely be a necessary pre-requisite for a 'good' implementation of the idea.
The idea...
The core idea is to use probes and long-range scanning equipment on ships to manage those probes and their data connection. But not probes and scanners as we currently see them in-game. Rather, a variety of special-purpose consumable probes**, and corresponding ships and ship equipment to use those probes and manage the data coming back from their respective surveys. It would be these probes that do the initial discovery of POIs for you. It would be these probes and scanners that provide an in-game justification for detailed logging. And it would be return data sets from the probes that give you, the pilot, a potentially more fun time since you'd be able to focus on piloting to those locations that your probes have identified as being potentially interesting. You get to fly to the good stuff, and not just fly AFK on auto-pilot to find new areas.

So yes, I want navigation data logging, but I wasn't ready to dig too deep into that because of the tie-in with this much larger gameplay feature I was planning on a more organized and formal write-up about. Cat's out of the bag now.... and maybe it needs its own topic thread for focus, but would you find the addition of more/better and more varied probes and scanners and sensor arrays and sensor buoys of interest?

** For example: mass spectrometer probes that can identify the material composition of a sector telling you where the platinum is, where the gold is, etc. Survey probes that can identify the large body objects in a sector to give the initial lay of the land (well, sector), EM probes that can find nebular regions and their oh-so-fun lightning storms, shorter range probes that can sound out the insides of those hollow asteroids and find the fricking entrances for you, or planetary survey probes that map a planet for you, or shorter range (a sector?) probes that find out if a whole lot of Red Ships are in the sector you're about to jump to, and about a dozen other probe types of all sizes and costs and uses that may or may not be Good Ideas(tm).

[Edited on 10-14-2010 by Aesir Rising]

[Aures]

Yeah I agree it sucks. An alternative to jump and eyeball such as your suggestion would be much better. I just care about automatically logging the sector designation of sectors I pass through because I am building a territory map. I am kind of interested in the shape of systems rather than just the objects they contain. So the ability to always capture the data I am interested in without having to do anything is of obvious interest to me.

I mentioned automatically logging all objects in sector range as an example of something that shouldn't happen. To me defining the major pitfalls of logging is one of the first things that should be considered. I think you agree since you have obviously put some thought into the matter. Sorry I got you to show your cards before you planned to.

BTW I never shot down your idea about having an auto-save. I think an option to enable an auto-save feature would be a good addition. I have been thinking about starting a dedicated thread based on your posts about the save system. I am planning to quote part of your description of how the save system should work since I couldn't put it better myself. What we have now is inconsistent and confusing. The points you raised deserve their own thread.

[Atollski]

Hi all,

Have an idea - you can directly manipulate the map log data file for your pilot while the game is running and it will update from the file whenever you open the map log. So... If you have a database that you store all the information you are interested in, you could make it populate this 'points of interest' file with locations from the database (or feed the data back onto the database)

So, if you had station ware prices recorded on a database, you could run a query on the database for best prices near to your location and it could plonk them straight into your map log, saving lots of scrolling etc.

file path (on my computer anyway): C:\\sw3dg\\EvochronMercenary\\maplog0.sw

Map Entry format:

Code: Select all

> X=290 Y=0 Z=0 SX=110 SY=47 SZ=0,Awesome test

[/b]

[Ravenfeeder]

I've floated the idea of probes being able to collect data, and transmit it back to sender, a few times - usually been ignored!

[Aesir Rising]

Never give up. Never surrender.