Questions about internet radio streaming

[Dibo]

Hi Fred,

First of all, congratulations for your work. 2 years has passed since I used UOS last time. You made a big step since then. I'm really impressed. I see that UOS has now support for internet radio streaming. This functionality was missing 2 years ago and I migrated to Bass library because it was major extension for my player. Now I'm consider back to UOS. I have few questions about web streaming:
1. Is it optimized? I mean, I see that it call TFPHTTPClient.Get() . Does it mean that it continuously download more data into memory than he needs for playing? Or such things are limited on server site and radio station send only small packets enough for play?
2. How it behave with bad connection? Does it have something like buffering and when network is really slow it can handle it instead of disconnect?
3. Some radio stations emits meta data (current playing title / artist). Is it possible to read it?

Regards

[fredvs]

Hello Dibo and welcome back.

Dibo wrote

1. Is it optimized? I mean, I see that it call TFPHTTPClient.Get() . Does it mean that it continuously download more data into memory than he needs for playing? Or such things are limited on server site and radio station send only small packets enough for play?

uos uses pipes for Radio-internet streaming. So it depends of the server site. For example if the Internet-radio is a real live radio (with speech in real time), the server sent only small packets to play. If the Internet-radio is more a "virtual radio" (I mean  a server with play-list and mp3/ogg/opus files), then the pipes will try to download all the faster it can, even more than what needed to play...

Dibo wrote

2. How it behave with bad connection? Does it have something like buffering and when network is really slow it can handle it instead of disconnect?

Yes, the pipes do that job.

Dibo wrote

3. Some radio stations emits meta data (current playing title / artist). Is it possible to read it?

Yes, uos reads the meta data too (but only for  mp3, ogg or opus files).

Maybe you may try conswebstream.lpi/prj,  simplewebplayer.lpi/prj and  simplewebplayer_fpgui.lpi/prj.

Fre;D

[fredvs]

Dibo wrote

3. Some radio stations emits meta data (current playing title / artist). Is it possible to read it?

Ha, ok, I think I get what you want, you mean when tag are updated by the server.
For update the meta data (IMHO and AFAIK), you need to re-connect to the url with a other player and do a "update Tag" something like this:

function UpdateTagFromURL(url_that_is_playing : string) : boolean;
begin
result := false;
  if uos_CreatePlayer(0) then
  if uos_AddFromURL(0,pchar(url_that_is_playing)) <> -1 then
  begin
  TagTitle:= uos_InputGetTagTitle(0,0);
  TagArtist:= uos_InputGetTagArtist(0,0);
  TagAlbum:= uos_InputGetTagAlbum(0,0);
  TagDate:= uos_InputGetTagDate(0,0);
  TagComment:= uos_InputGetTagComment(0,0);
  TagTag:= uos_InputGetTagTag(0,0);
  uos_Stop(0);
  result := true;
  end;
end;

Fre;D

[Dibo]

Yes, this is what I meant :) . So I need to do periodically

[fredvs]

Dibo wrote

Yes, this is what I meant :) . So I need to do periodically

Yes, for mp3 audio-streaming, it is what i see now.
With a timer or a button.

After a check-up in mpg123 lib, there is that method:

mpg123_meta_check()

I never use it, maybe it works for web-streaming too, so no need to create a other player.
Have to check.

For opus files it is possible to update it automatic when the server change the meta-infos (not tested yet).

Fre;D

[fredvs]

Dibo wrote

3. Some radio stations emits meta data (current playing title / artist). Is it possible to read it?

Hello.

Added uos_InputUpdateTag(..) in last commit 0b9de37..b028a0d.

It updates the tag-infos of the running player but must be done periodic to check if infos were changed.
So far, I did not find a way to do it automatic. (Getting a message from the server saying that the tags were updated).

Fre;D

[Dibo]

Here is something interesting, epsecialy answer:
http://stackoverflow.com/questions/6061057/developing-the-client-for-the-icecast-server
http://www.smackfu.com/stuff/programming/shoutcast.html

So it could be basically done in HTTP Thread by adding 'Icy-MetaData' header and counting received bytes. I'll try to test it over the long weekend. Thanks Fred!

[fredvs]

Dibo wrote

Here is something interesting, epsecialy answer:
http://stackoverflow.com/questions/6061057/developing-the-client-for-the-icecast-server
http://www.smackfu.com/stuff/programming/shoutcast.html

There are some progress with uos + icecast-shout-server.
The tools are ready (uos_shout.pas + uos_opus.pas and demo simplewebserver.pas).

I was able to do it work locally (but not with a real server across internet --> no access).

Maybe some infos from uos_shout.pas could be useful.

I have very difficult internet access so it is not easy to try it in real.

Fre;D

[Dibo]

I was thinking about modify TThreadHttpGetter in uos_httpgetthread similiar like they did it here (TAuHTTPThread is equivalent for your TThreadHttpGetter):
https://github.com/astoeckel/audorra/blob/master/src/protocols/AuHTTP.pas
So in execute method add:

  repeat
  try
    Http.RequestHeaders.Clear;
    Http.RequestHeaders.Add('Accept: icy-metadata:1');
    Http.Get(URL, FOutStream);
  except

... get response icy-metaint header, save and handle it in FOutStream (bytes counting). I'm not sure if this header should be send for each Get() but that is a matter of tests. I think that all radio stations which "emits" metadata just use this icy protocol. StreamTitle='title of the song' - this tag is very familiar for me, I parsed it in bass library for internet radio streaming and also in Qt Media Player. So this protocol is very common for radio stations and UOS is very close to have it too :)

[fredvs]

Hello Dibo.

Of course your commits are welcome.
Huh, do you know a url of web-radio that sent meta-data ?

Fre;D

[Dibo]

fredvs wrote

Huh, do you know a url of web-radio that sent meta-data ?

Yep, Radio Paradise:
https://www.radioparadise.com/rp_2.php?#name=Listen&file=links

[Dibo]

Just did quick tests:

    Http.RequestHeaders.Clear;
    Http.RequestHeaders.Add('Icy-MetaData:1');
    Http.OnHeaders := @Headers;
    Http.OnDataReceived := @Data;
    Http.Get(URL, FOutStream);    

Indeed after that I'm getting expected header in OnHeaders event (icy-metaint):

Content-Type: audio/mpeg
icy-br:192
icy-genre:eclectic rock
icy-name:Radio Paradise - DJ-mixed modern & classic rock, world, electronica & more - info: radioparadise.com
icy-notice1:<BR>This stream requires Winamp<BR>
icy-notice2:SHOUTcast Distributed Network Audio Server/Linux v1.9.8<BR>
icy-pub:0
icy-url:http://www.radioparadise.com
Server: Icecast 2.3.3-kh9
Cache-Control: no-cache
Pragma: no-cache
Expires: Mon, 26 Jul 1997 05:00:00 GMT
icy-metaint:32768

So now I think it is just a matter read tag in Http.OnDataReceived := @Data at every position 32768 but I'm not sure how to do it since all content is directly flushed into pipe. Maybe Http.Get() should read first into temporary buffer and then send to FOutStream. I'll test more in the evening

[Dibo]

Also, when Http.RequestHeaders.Add('Icy-MetaData:1'); is added then I'm getting weird console warnings:

Press a key to exit...
Note: Illegal Audio-MPEG-Header 0xc3851009 at offset 1519.
Note: Trying to resync...
Note: Skipped 234 bytes in input.

Warning: Big change from first (MPEG version, layer, rate). Frankenstein stream?
Note: Illegal Audio-MPEG-Header 0x8134008d at offset 33100.
Note: Trying to resync...
Note: Skipped 129 bytes in input

That is probably because string tag was received and mpeg123 tried play it :P . So seems that such tag chunk has to be cut off from content before sending to pipe anyway

[fredvs]

Dibo wrote

Warning: Big change from first (MPEG version, layer, rate). Frankenstein stream?
Note: Illegal Audio-MPEG-Header 0x8134008d at offset 33100.
Note: Trying to resync...
Note: Skipped 129 bytes in input
That is probably because string tag was received and mpeg123 tried play it :P . So seems that such tag chunk has to be cut off from content before sending to pipe anyway

Maybe you could compile new mpg123 source (https://www.mpg123.de).
For Unix system it is easy, but for Windows, it is a other game.
Luckily, they give pre-compiled win libraries too :

https://www.mpg123.de/download/win32/1.24.0/
https://www.mpg123.de/download/win64/1.24.0/

Fre;D

[Dibo]

Maybe you could compile new mpg123 source (https://www.mpg123.de).
For Unix system it is easy, but for Windows, it is a other game.

Just did this now but still same warnings. It is related to Icy-MetaData:1 header. Meta data should be cut off before sending to mpg123. Do you have idea how to extract this? I'm not familiar with UOS source and pipes

[fredvs]

Dibo wrote

Meta data should be cut off before sending to mpg123. Do you have idea how to extract this? I'm not familiar with UOS source and pipes.

Once uos_AddFromURL() is executed, there is only one call in the main-thread loop:

mpg123_read(StreamIn[x].Data.HandleSt, @StreamIn[x].Data.Buffer[0],
StreamIn[x].Data.wantframes, StreamIn[x].Data.outframes);

So it is mpg123 who reads the data (and uos uses pipes to store those data), according of the handle assigned at creation.
OK, I must think about it.

I jump into the problem asap and write you asap.

A other solution is to not use pipes, but only a Tmemorystream.
But it is a few sad because piping works good.

Pipes works +- like memory stream (without possibility to seek).

A other solution, that I have used many times (with always sucess) is to... ask it to mpg123 forum.
But before to do this, I have to perfectly understand the problem, then I will ask it to Thomas.

Fre;D

[Dibo]

fredvs wrote

A other solution is to not use pipes, but only a Tmemorystream.
But it is a few sad because piping works good.

I agree, better don't break thing which works fine and find another workaround

[fredvs]

Dibo wrote

    Http.RequestHeaders.Add('Accept: icy-metadata:1');
 

Ha, I think I get something:
In library mpg123 there is that method:

mpg123_icy ( mpg123_handle *  mh, char **icy_meta)

See here:  https://www.mpg123.de/api/group__mpg123__metadata.shtml 

It is translated already in uos_mpg123.pas:

mpg123_icy = function(mh: Tmpg123_handle; var icy_meta: PPChar): integer; cdecl;

and

MPG123_ICY_INTERVAL = 10;

OK, I think we get it, it will work.
So maybe that after execute mpg123_icy(mphandle, my_icy_data) , each 10 seconds my_icy_data will be updated.
 
Must try.

I will add a uos_InputEnableICY().

Fre;D

[fredvs]

And here:

https://www.mpg123.de/api/group__mpg123__init.shtml

----> mpg123_feature_set --->  MPG123_FEATURE_PARSE_ICY  // ICY support

So executing the method: -->

mpg123_feature(MPG123_FEATURE_PARSE_ICY) --->

Must do the trick

Fre;D

[fredvs]

fredvs wrote

mpg123_feature(MPG123_FEATURE_PARSE_ICY) --->

Must do the trick

No, it only crash the application ;-(

OK, I stop tonight, maybe tomorrow will find better inspiration.

Fre;D