Re: [liblo] maximum size of message

On Mon, Sep 29, 2014 at 3:43 AM,  <to...@tr...> wrote:
> Hi Steve
>
> On Sun, September 28, 2014 23:50, Stephen Sinclair wrote:
>> Hi Tom,
>>
>>
>> I'm not sure if the misunderstanding is on my side or yours, but the
>> UDP max message size is due to how UDP works, not due to anything
>> regarding OSC or liblo.
>>
>
> yes, i think so far i can follow. this is IIUC the reason why in the
> fixmax branch there is a new constant LO_MAX_UDP_MSG_SIZE that reflects
> the limit given by network specs and the "old" LO_MAX_MSG_SIZE with half
> the size is kept for compatibility.
>
> my question was, how could a program that relies on liblo find out what
> max UDP size the *currently installed* liblo supports? If this is a
> version that still uses LO_MAX_MSG_SIZE then it's limited to 32768
> (sending and receiving) and if it's a newer version it could be 65535. How
> could that be determined at runtime?

Well, unfortunately since there is no previous API for this constant
in the current version of liblo, of course it's impossible to
determine it with a function call.  However, if the function
lo_server_max_msg_size() exists in the library (determined e.g. using
autoconf AC_CHECK_FUNC), then you can call it with,

int prev_max_size = lo_server_max_msg_size(s, 0);

This will return the previous maximum message size without changing
it.  If it returns -1, message size is unlimited.  At least this is
the API that I am proposing in the fixmax branch.  Comments are
welcome.

If this function doesn't exist, the max should be assumed to be
LO_MAX_MSG_SIZE = 32k.  (I could add this comment to the header
perhaps.)


>> I'm won't say right out that it's wrong, but It's a little unusual to
>> be sending audio data via OSC messages.  Usually OSC is considered a
>> control protocol, like MIDI.  Maybe you could give us the bigger picture
>> of what you are trying to accomplish?
>>
>
> i keep hearing that, "OSC is for control data" :) i agree it's (not just)
> for that. if you're interested in the program, it's all on github under
> audio-rxtx but this wasn't the focus of my mail (i just mentioned a nice
> side effect once we have 65k max UDP which again was a side-effect of
> asking for larger TCP).
> MIDI has similarities, but not only for control data. sysex was often used
> to transfer sample/wave data so that's nothing new or weird, but it could
> be done say in 1 MB blocks using TCP with modern OSC clients today. my 2
> cent :)

It's true that it can be used to send any old data using blobs, but
fundamentally OSC is a message-based protocol.  The impact is that
afaik no known OSC implementation supports unlimited sized message
reception.  You could consider this an "underdefined" bug in the OSC
spec if you wish, but basically cross-implementation compatibility
would require all OSC implementations to support some kind of
incremental interface. (Like callbacks for example.)  Since this isn't
really the current state of things, using very large messages won't be
well-supported across implementations.  Additionally, as IOhannes
mentions, the semantics of bundles really doesn't agree well with the
idea.  This discussion could be taken to the OSC_dev list if you want
to continue it.

I would strongly suggest that for large transfers you open a dedicated
TCP (or HTTP) connection, in parallel to your OSC port.  Note that
this would also avoid blocking your control data while transferring
large amounts of audio/sysex data for example.  Don't try to pound in
screws with a hammer.

My recommendation is that OSC messages stay under 64k for
compatibility reasons, as it is basically an ad-hoc convention.
However, I don't mind expanding liblo to support larger maximums on
explicit request from the application code.

... thinking on the sysex thing further.. probably system state should
be transferred in a more semantic way than a single OSC blob.  i.e., a
series of messages defining the (timestamped) value of each (named)
variable.  TCP is appropriate for this, since UDP messages may get
lost.


Steve