civil-devel Mailing List for Civil

On Thu, 8 Jul 2004, Mike Earl wrote:

> Let's say A,B,C are on one side and N,M,O on another.
>
> If we determine that A sees N, do we need to check B & C? (I think
> not?)  It might even make sense (esp. on large maps with many units) to
> evaluate distances to all enemy units first, sort them, and check to see
> if we're seen by the closest ones first.

I don't think A seeing N means that B and C automatically see A. It's 
a good idea, but automatic visibility propagation isn't very 
realistic, IMHO.

> The A->N LoS will share most of the calculation (the expensive part?)
> with N->A; you may want to do them at the same time.

Sounds good!

On Thu, 2004-07-08 at 03:21, Jan Ekholm wrote:
> Well, this is my grand vision of how the LOS should work. It's obviously
> flawed in several serious ways, so please let me know where the major
> faults are, or just silently fix my goofs in CVS after I commit the stuff.

No, sounds good. I can suggest a couple of (obvious?) optimizations that
may win if N,M get big:

> 1. all own units are always visible.
> 
> 2. the server does all the calculations and sends out SetVisibility
> actions when visibility for a unit changes.
> 
> 3. the server iterates over all N*M units and checks LOS between each pair
> of units. The general LOS value that the check gives is modified depending
> on the units. For example if A is hidden and B moves on an open field
> nearby there is a clear LOS from A->B and B->A. However, A being hidden
> means that A sees B but not the reverse. These calculations are left out
> for now.

Let's say A,B,C are on one side and N,M,O on another.

If we determine that A sees N, do we need to check B & C? (I think
not?)  It might even make sense (esp. on large maps with many units) to
evaluate distances to all enemy units first, sort them, and check to see
if we're seen by the closest ones first.

The A->N LoS will share most of the calculation (the expensive part?)
with N->A; you may want to do them at the same time.

With any luck you end up with a lot less than N*M/2.


> The actual implementation is fairly simple, but in order to not send out
> stupid amounts of data some care has to be taken. The server needs to
> store the visibility the unit had during the last LOS update (the unit's
> internal visibility flag) and a set of new visibility flags for the LOS
> update. All units start as not visible at first. Each unit that an
> enemy unit sees is marked as visible in the new set of flags. A unit that
> is not seen triggers no update. In the end one compares the new set of
> flags to the units' last visibility, and if they differ then a
> SetVisibility is sent out with the new visibility status. After this the
> new flag set is stored in the units' internal visibility flags.

Yes, sounds good.  My only suggestion (as above) is to consider doing
A->N and N->A at same time, and not to bother looking further at LoS to
a unit once you know he's visible. 

- mikee

On Thu, 8 Jul 2004, Jan Ekholm wrote:

>
> Hm, quite simple, so it must be broken somehow?

No, I think this is exactly how it was with the old action system.

/M

Well, this is my grand vision of how the LOS should work. It's obviously
flawed in several serious ways, so please let me know where the major
faults are, or just silently fix my goofs in CVS after I commit the stuff.

1. all own units are always visible.

2. the server does all the calculations and sends out SetVisibility
actions when visibility for a unit changes.

3. the server iterates over all N*M units and checks LOS between each pair
of units. The general LOS value that the check gives is modified depending
on the units. For example if A is hidden and B moves on an open field
nearby there is a clear LOS from A->B and B->A. However, A being hidden
means that A sees B but not the reverse. These calculations are left out
for now.

The actual implementation is fairly simple, but in order to not send out
stupid amounts of data some care has to be taken. The server needs to
store the visibility the unit had during the last LOS update (the unit's
internal visibility flag) and a set of new visibility flags for the LOS
update. All units start as not visible at first. Each unit that an
enemy unit sees is marked as visible in the new set of flags. A unit that
is not seen triggers no update. In the end one compares the new set of
flags to the units' last visibility, and if they differ then a
SetVisibility is sent out with the new visibility status. After this the
new flag set is stored in the units' internal visibility flags.

Hm, quite simple, so it must be broken somehow?


-- 
           "Right, you bastards, you're... you're geography"
                                         -- Terry Pratchett, Guards! Guards!

On Wed, 7 Jul 2004, Marcus Alanen wrote:

>On Wed, 7 Jul 2004, Jan Ekholm wrote:
>
>> You are probably right. So be it then. I'll add a small module for that
>> purpose. Any ideas for how to avoid doing N*N LOS checks (each unit with
>> each unit)? Ok, actually all friendlies against all enemies, so basically
>> N*(N/2) comparisons.
>
>No, but we could start by just trying the N*N/2 comparisons and see if
>it is good enough.

Ok, it doesn't have to be done every second either. Maybe once every 4s
player time (once every 30s simulation time)? Or something like that.

>How have we decided previously: if A sees Z, does Z always see A? I'd
>myself wouldn't want this, it'd enable A to hide in the forest close
>to a field, wait for Z to come close and then start shooting. Then Z
>would also immediately see A due to A shooting at Z. Sound ok?

Yes, LOS has to be asymmetric. A hiding small unit can see a lot of other
units that can't see it. So A seing B doesn't imply that B sees A. Firing
on a unit would reveal the attacker immediately to the target.

-- 
  "You can't trample infidels when you're a tortoise. I mean, all you could
                  do is give them a meaningful look."
                                              -- Terry Pratchett, Small Gods

On Wed, 7 Jul 2004, Jan Ekholm wrote:

> You are probably right. So be it then. I'll add a small module for that
> purpose. Any ideas for how to avoid doing N*N LOS checks (each unit with
> each unit)? Ok, actually all friendlies against all enemies, so basically
> N*(N/2) comparisons.

No, but we could start by just trying the N*N/2 comparisons and see if 
it is good enough.

How have we decided previously: if A sees Z, does Z always see A? I'd 
myself wouldn't want this, it'd enable A to hide in the forest close 
to a field, wait for Z to come close and then start shooting. Then Z 
would also immediately see A due to A shooting at Z. Sound ok?

On Tue, 6 Jul 2004, Marcus Alanen wrote:

>On Tue, 6 Jul 2004, Jan Ekholm wrote:
>
>> The LOS outline may be off by default nowadays. As for the normal who sees
>> who stuff, there is still some thinking to do. Should the clients
>> themselves evaluate who they see, or is it up to the server to tell the
>> clients what that client's units see right now? The latter is better, but
>> requires more bandwidth. Not that there actually would be that many
>> changes per second, I think.
>
>I'd say server. I don't think the bandwidth bill will be too high. It
>could just send the changes in sight, not complete sight information.
>Actually I think we already send the same stuff many
>times with ChangeModifiersAct.

You are probably right. So be it then. I'll add a small module for that
purpose. Any ideas for how to avoid doing N*N LOS checks (each unit with
each unit)? Ok, actually all friendlies against all enemies, so basically
N*(N/2) comparisons.

>> But that map already knows about the terrain, it's basically just a
>> different "cost" for each terrain type. Water has no cost at all while
>> woods have a big cost.
>
>Yes, this is what I meant.
>
>/M
>

-- 
    There were no public health laws in Ankh-Morpork. It would be like
                installing smoke detectors in Hell.
                                            -- Terry Pratchett, Feet of Clay

On Tue, 6 Jul 2004, Jan Ekholm wrote:

> The LOS outline may be off by default nowadays. As for the normal who sees
> who stuff, there is still some thinking to do. Should the clients
> themselves evaluate who they see, or is it up to the server to tell the
> clients what that client's units see right now? The latter is better, but
> requires more bandwidth. Not that there actually would be that many
> changes per second, I think.

I'd say server. I don't think the bandwidth bill will be too high. It 
could just send the changes in sight, not complete sight information.
Actually I think we already send the same stuff many 
times with ChangeModifiersAct.

> But that map already knows about the terrain, it's basically just a
> different "cost" for each terrain type. Water has no cost at all while
> woods have a big cost.

Yes, this is what I meant.

/M

On Tue, 6 Jul 2004, Marcus Alanen wrote:

>>>- broken LOS
>> In what way? Units that aren't seing each other is something I've also
>
>It doesn't show the LOS outline, and the nobody seems to tell the units
>that they see new enemies, I think. This was the work of old/change_los.py.

The LOS outline may be off by default nowadays. As for the normal who sees
who stuff, there is still some thinking to do. Should the clients
themselves evaluate who they see, or is it up to the server to tell the
clients what that client's units see right now? The latter is better, but
requires more bandwidth. Not that there actually would be that many
changes per second, I think.

>> noticed. What does the LOS outline show, or is it broken? Btw, I think
>> we'll not allow the players to use the LOS outline at all, it just doesn't
>> seem useful. Something that could be used instead is to let the user check
>> LOS to given positions on the map, or just trace LOS from a unit to the
>> current cursor position.
>
>Hm, dunno, perhaps not overly realistic with so complete information.
>But, let's just fix it first.

Oh, no, are you getting realistic when there is good brainstorming and
vague ideas being thrown out?

>>>- shortest-path is quite broken, unfortunately.
>> You mean the pathfinding? I think it should be altered to try to use the
>> same image map as the LOS uses.
>
>This would assume LOS values are equal to path cost values, so I don't
>think it can use exactly the same (think lakes: good LOS, bad path
>cost), but perhaps a similar image map.

But that map already knows about the terrain, it's basically just a
different "cost" for each terrain type. Water has no cost at all while
woods have a big cost.

-- 
                        "Bingeley bingeley beep!"
               -- The Personal Disorganizer, Terry Pratchett in Feet of Clay

>>- broken LOS
> In what way? Units that aren't seing each other is something I've also

It doesn't show the LOS outline, and the nobody seems to tell the units 
that they see new enemies, I think. This was the work of old/change_los.py.

> noticed. What does the LOS outline show, or is it broken? Btw, I think
> we'll not allow the players to use the LOS outline at all, it just doesn't
> seem useful. Something that could be used instead is to let the user check
> LOS to given positions on the map, or just trace LOS from a unit to the
> current cursor position.

Hm, dunno, perhaps not overly realistic with so complete information. 
But, let's just fix it first.

>>- shortest-path is quite broken, unfortunately.
> You mean the pathfinding? I think it should be altered to try to use the
> same image map as the LOS uses.

This would assume LOS values are equal to path cost values, so I don't 
think it can use exactly the same (think lakes: good LOS, bad path 
cost), but perhaps a similar image map.

On Mon, 5 Jul 2004, Marcus Alanen wrote:

>Jan Ekholm wrote:
>> On Mon, 5 Jul 2004, Marcus Alanen wrote:
>>
>>
>>>Hm. I haven't looked too much into it, but we only do
>>>"del scenario.info.units[self.id]" in units.py.
>>>
>>>An easy solution would be to just iterate using .values(), which
>>>creates a new temporary list. Then check if the unit is really dead,
>>>if it isn't, continue with its plans. ?
>> Didn't know that .itervalues() disallowed changing the collection. Hm,
>
>I guess it's there to make stuff safer to use. Makes some algorithms a
>bit more awkward, though.
>
>> I think it should be justok to use .values(), as the unit isn't really
>> used after that anymore, it gets removed from scenario.info.units and
>> shouldn't be referenced anymore.
>
>I tried it out, and it seems to have worked fine. Units die and nothing
>crashes.

Good.

>Bugs so far:
>- broken LOS

In what way? Units that aren't seing each other is something I've also
noticed. What does the LOS outline show, or is it broken? Btw, I think
we'll not allow the players to use the LOS outline at all, it just doesn't
seem useful. Something that could be used instead is to let the user check
LOS to given positions on the map, or just trace LOS from a unit to the
current cursor position.

>- shortest-path is quite broken, unfortunately.
>- ?

You mean the pathfinding? I think it should be altered to try to use the
same image map as the LOS uses.

>Oh, by the way. It'll be fun trying to make the game playable, because
>the current going rate for one's army seems to be "slaughter". :-)

No, no, your tactics suck. ;)

-- 
You've got a lot of time for abstract thought when you've got your hand
    stuck up a dead badger.
                                 -- Terry Pratchett, Johnny and the Bomb

Jan Ekholm wrote:
> On Mon, 5 Jul 2004, Marcus Alanen wrote:
> 
> 
>>Hm. I haven't looked too much into it, but we only do
>>"del scenario.info.units[self.id]" in units.py.
>>
>>An easy solution would be to just iterate using .values(), which
>>creates a new temporary list. Then check if the unit is really dead,
>>if it isn't, continue with its plans. ?
> Didn't know that .itervalues() disallowed changing the collection. Hm,

I guess it's there to make stuff safer to use. Makes some algorithms a 
bit more awkward, though.

> I think it should be justok to use .values(), as the unit isn't really
> used after that anymore, it gets removed from scenario.info.units and
> shouldn't be referenced anymore.

I tried it out, and it seems to have worked fine. Units die and nothing 
crashes.

Bugs so far:
- broken LOS
- shortest-path is quite broken, unfortunately.
- ?

Oh, by the way. It'll be fun trying to make the game playable, because 
the current going rate for one's army seems to be "slaughter". :-)

Jan Ekholm wrote:

>>Some *.py files use non-ascii characters, which emits a warning in
>>python2.3 if no character encoding is declared. I'll fix them as well.
> Yeah, I've fixed a few of those as well, but some of them I can't seem to
> find. For some file I had to remove all whitespace on a line and reinsert
> it before the warning disappeared. I have no idea what triggers it... If

Hm, I think I had some old checkout, since now they are gone. Anyway, 
they were no-break space characters. In ISO-8859-1, octal 240. I guess 
some editor has put them there when hitting shift-space or some other 
combination.

/Marcus

On Mon, 5 Jul 2004, Marcus Alanen wrote:

>Hm. I haven't looked too much into it, but we only do
>"del scenario.info.units[self.id]" in units.py.
>
>An easy solution would be to just iterate using .values(), which
>creates a new temporary list. Then check if the unit is really dead,
>if it isn't, continue with its plans. ?

Didn't know that .itervalues() disallowed changing the collection. Hm,
there *may* be some other similar buglets in there.

I think it should be justok to use .values(), as the unit isn't really
used after that anymore, it gets removed from scenario.info.units and
shouldn't be referenced anymore.

-- 
You've got a lot of time for abstract thought when you've got your hand
    stuck up a dead badger.
                                 -- Terry Pratchett, Johnny and the Bomb

Hm. I haven't looked too much into it, but we only do
"del scenario.info.units[self.id]" in units.py.

An easy solution would be to just iterate using .values(), which 
creates a new temporary list. Then check if the unit is really dead, 
if it isn't, continue with its plans. ?


server: oops, something went wrong. Dumping brain contents:
---------------------------------------------------------------------------
Traceback (most recent call last):
   File "./civil-server.py", line 262, in ?
Unit.applyDamage: id: 57, 4 killed, 0 left
Unit.applyDamage: unit 57 is destroyed
Unit.applyDamage: 66 ok, 1 destroyed
Unit.applyDamage: 57: Headquarter Aspen Cavalry HQ destroyed
     main ()
   File "./civil-server.py", line 253, in main
     server.globals.engine.mainLoop ()
   File "/export/nobackup/maalanen/civil/src/server/engine/engine.py", 
line 106, in mainLoop
     self.__update ()
   File "/export/nobackup/maalanen/civil/src/server/engine/engine.py", 
line 152, in __update
     for unit in scenario.info.units.itervalues ():
RuntimeError: dictionary changed size during iteration

>>> You can get inspiration from mdk package if you want:
>>> http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/contrib-SPECS/civil/

Guillaume, I tried again and things seem to be ok now. Thanks for the 
link though, I see you have some good fixes there as well. I've just 
added a build-dependency on python2.3 because that's what my work 
computer has, but apparently this can be fixed much smoother.

> Yeah, there isn't one yet. I never got as far as doing it, I still have a
> few minor things I'd like to do for 0.83. I got caught up with a shovel
> trying to get the Baghdad-like landscape on our garden to resemble the
> Gardens of Eden. Not there yet... :)

OK, no need to rush for me, you seem to be the only active one 
anyway.

Some *.py files use non-ascii characters, which emits a warning in 
python2.3 if no character encoding is declared. I'll fix them as well.

/M

On Sun, 4 Jul 2004, Marcus Alanen wrote:

>Guillaume Rousse wrote:
>> Marcus Alanen wrote:
>>
>>> Hi, I've been trying to make the RPM packages for civil. I have some
>>> problems with getting ccivil.so copied into the install (probably
>>> rusty rpm skills...), but I'll see what I can do in the next day(s) or
>>> so.
>>
>> You can get inspiration from mdk package if you want:
>> http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/contrib-SPECS/civil/
>>
>> I'm just waiting for official 0.83 release to update it.
>
>Chakie,
>
>I've finally done something for this, so I am able to build the RPM
>packages on a Fedora Core 2 at work.
>
>However, there doesn't seem to be any 0.83 in CVS? All I see is
>CIVIL-0_82-3.

Yeah, there isn't one yet. I never got as far as doing it, I still have a
few minor things I'd like to do for 0.83. I got caught up with a shovel
trying to get the Baghdad-like landscape on our garden to resemble the
Gardens of Eden. Not there yet... :)

I'll do it this week.

--
  Shadwell hated all southerners and, by inference, was standing at the
  North Pole.
                           -- Terry Pratchett & Neil Gaiman, Good Omens

Guillaume Rousse wrote:
> Marcus Alanen wrote:
> 
>> Hi, I've been trying to make the RPM packages for civil. I have some 
>> problems with getting ccivil.so copied into the install (probably 
>> rusty rpm skills...), but I'll see what I can do in the next day(s) or 
>> so.
> 
> You can get inspiration from mdk package if you want:
> http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/contrib-SPECS/civil/
> 
> I'm just waiting for official 0.83 release to update it.

Chakie,

I've finally done something for this, so I am able to build the RPM 
packages on a Fedora Core 2 at work.

However, there doesn't seem to be any 0.83 in CVS? All I see is 
CIVIL-0_82-3.

/Marcus

Marcus Alanen wrote:
> Hi, I've been trying to make the RPM packages for civil. I have some 
> problems with getting ccivil.so copied into the install (probably 
> rusty rpm skills...), but I'll see what I can do in the next day(s) or 
> so.
You can get inspiration from mdk package if you want:
http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/contrib-SPECS/civil/

I'm just waiting for official 0.83 release to update it.