An alternative would be supporting an M4B with two tracks, where the Dolby audio is available to switch to on supported devices.

Have you confirmed that we can transcode the file using the ffmpeg version bundled with Audiobookshelf? Some formats and codecs are not supported by a distributable build due to licensing restrictions and must be built by the end user, such as with widevine and xHE-ACC

#4236

It is decodable with the linux binary of ffmpeg 5.1 here https://ffbinaries.com/downloads

FFMPEG reverse engineered their own decoder in 2009 and have included it as a default build option since 0.5.

Can you please check which mimetype ABS thinks this is?

It should be part of the API response under audioFiles

What is a user-facing way to query that?

Press the "Probe Audio File" button

{
"programs": [],
"streams": [
{
"index": 0,
"codec_name": "eac3",
"codec_long_name": "ATSC A/52B (AC-3, E-AC-3)",
"profile": "Dolby Digital Plus + Dolby Atmos",
"codec_type": "audio",
"codec_tag_string": "ec-3",
"codec_tag": "0x332d6365",
"sample_fmt": "fltp",
"sample_rate": "48000",
"channels": 6,
"channel_layout": "5.1(side)",
"bits_per_sample": 0,
"initial_padding": 0,
"id": "0x1",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/48000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 1391494608,
"duration": "28989.471000",
"bit_rate": "768000",
"nb_frames": "905921",
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"non_diegetic": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"language": "und",
"handler_name": "SoundHandler",
"vendor_id": "[0][0][0][0]"
},
"side_data_list": [
{
"side_data_type": "Audio Service Type",
"service_type": 0
}
]
},
{
"index": 1,
"codec_name": "bin_data",
"codec_long_name": "binary data",
"codec_type": "data",
"codec_tag_string": "text",
"codec_tag": "0x74786574",
"id": "0x3",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28989471,
"duration": "28989.471000",
"nb_frames": "19",
"extradata_size": 43,
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"non_diegetic": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"language": "eng",
"handler_name": "SubtitleHandler"
}
},
{
"index": 2,
"codec_name": "mjpeg",
"codec_long_name": "Motion JPEG",
"profile": "Progressive",
"codec_type": "video",
"codec_tag_string": "[0][0][0][0]",
"codec_tag": "0x0000",
"width": 2400,
"height": 2400,
"coded_width": 2400,
"coded_height": 2400,
"closed_captions": 0,
"film_grain": 0,
"has_b_frames": 0,
"sample_aspect_ratio": "1:1",
"display_aspect_ratio": "1:1",
"pix_fmt": "yuvj420p",
"level": -99,
"color_range": "pc",
"color_space": "bt470bg",
"chroma_location": "center",
"refs": 1,
"id": "0x0",
"r_frame_rate": "90000/1",
"avg_frame_rate": "0/0",
"time_base": "1/90000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 2609052390,
"duration": "28989.471000",
"bits_per_raw_sample": "8",
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 1,
"timed_thumbnails": 0,
"non_diegetic": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
}
}
],
"chapters": [
{
"id": 0,
"time_base": "1/1000",
"start": 0,
"start_time": "0.000000",
"end": 671458,
"end_time": "671.458000",
"tags": {
"title": "Foreword"
}
},
{
"id": 1,
"time_base": "1/1000",
"start": 671458,
"start_time": "671.458000",
"end": 1722333,
"end_time": "1722.333000",
"tags": {
"title": "Chapter 1: The Dursleys"
}
},
{
"id": 2,
"time_base": "1/1000",
"start": 1722333,
"start_time": "1722.333000",
"end": 4389124,
"end_time": "4389.124000",
"tags": {
"title": "Chapter 2: The Potters and the Malfoys"
}
},
{
"id": 3,
"time_base": "1/1000",
"start": 4389124,
"start_time": "4389.124000",
"end": 7516832,
"end_time": "7516.832000",
"tags": {
"title": "Chapter 3: Wands"
}
},
{
"id": 4,
"time_base": "1/1000",
"start": 7516832,
"start_time": "7516.832000",
"end": 9246790,
"end_time": "9246.790000",
"tags": {
"title": "Chapter 4: How the Wizarding World Works"
}
},
{
"id": 5,
"time_base": "1/1000",
"start": 9246790,
"start_time": "9246.790000",
"end": 9839706,
"end_time": "9839.706000",
"tags": {
"title": "Chapter 5: The Tri-Wizard Schools"
}
},
{
"id": 6,
"time_base": "1/1000",
"start": 9839706,
"start_time": "9839.706000",
"end": 10548122,
"end_time": "10548.122000",
"tags": {
"title": "Chapter 6: Alternate Histories"
}
},
{
"id": 7,
"time_base": "1/1000",
"start": 10548122,
"start_time": "10548.122000",
"end": 11825330,
"end_time": "11825.330000",
"tags": {
"title": "Chapter 7: Dark Creatures"
}
},
{
"id": 8,
"time_base": "1/1000",
"start": 11825330,
"start_time": "11825.330000",
"end": 12758621,
"end_time": "12758.621000",
"tags": {
"title": "Chapter 8: Animal Magic"
}
},
{
"id": 9,
"time_base": "1/1000",
"start": 12758621,
"start_time": "12758.621000",
"end": 13451121,
"end_time": "13451.121000",
"tags": {
"title": "Chapter 9: Wizarding Pastimes"
}
},
{
"id": 10,
"time_base": "1/1000",
"start": 13451121,
"start_time": "13451.121000",
"end": 14778996,
"end_time": "14778.996000",
"tags": {
"title": "Chapter 10: Fame in the Wizarding World"
}
},
{
"id": 11,
"time_base": "1/1000",
"start": 14778996,
"start_time": "14778.996000",
"end": 15685496,
"end_time": "15685.496000",
"tags": {
"title": "Chapter 11: Wizarding Transport"
}
},
{
"id": 12,
"time_base": "1/1000",
"start": 15685496,
"start_time": "15685.496000",
"end": 16301496,
"end_time": "16301.496000",
"tags": {
"title": "Chapter 12: The Journey to Hogwarts"
}
},
{
"id": 13,
"time_base": "1/1000",
"start": 16301496,
"start_time": "16301.496000",
"end": 16698037,
"end_time": "16698.037000",
"tags": {
"title": "Chapter 13: Sorting"
}
},
{
"id": 14,
"time_base": "1/1000",
"start": 16698037,
"start_time": "16698.037000",
"end": 23649037,
"end_time": "23649.037000",
"tags": {
"title": "Chapter 14: The Hogwarts Curriculum"
}
},
{
"id": 15,
"time_base": "1/1000",
"start": 23649037,
"start_time": "23649.037000",
"end": 24808620,
"end_time": "24808.620000",
"tags": {
"title": "Chapter 15: Magical Artifacts"
}
},
{
"id": 16,
"time_base": "1/1000",
"start": 24808620,
"start_time": "24808.620000",
"end": 27481286,
"end_time": "27481.286000",
"tags": {
"title": "Chapter 16: Hogwarts Alumni"
}
},
{
"id": 17,
"time_base": "1/1000",
"start": 27481286,
"start_time": "27481.286000",
"end": 28754244,
"end_time": "28754.244000",
"tags": {
"title": "Chapter 17: Exploring Hogwarts"
}
},
{
"id": 18,
"time_base": "1/1000",
"start": 28754244,
"start_time": "28754.244000",
"end": 28989471,
"end_time": "28989.471000",
"tags": {
"title": "End Credits"
}
}
],
"format": {
"filename": "/mnt/abs/Audiobook/J.K. Rowling/From the Wizarding Archive/From the Wizarding Archive.m4b",
"nb_streams": 3,
"nb_programs": 0,
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
"format_long_name": "QuickTime / MOV",
"start_time": "0.000000",
"duration": "28989.471000",
"size": "2783653190",
"bit_rate": "768183",
"probe_score": 100,
"tags": {
"major_brand": "mp42",
"minor_version": "512",
"compatible_brands": "mp42dby1iso2mp41",
"title": "From the Wizarding Archive",
"artist": "J.K. Rowling; Evanna Lynch - foreword",
"album_artist": "J.K. Rowling; Evanna Lynch - foreword",
"composer": "Evanna Lynch; Hugh Quarshie; Finlay Robertson; Lara Sawalha",
"album": "From the Wizarding Archive: Curated Writing from the World of Harry Potter",
"date": "2024",
"encoder": "cm4b",
"comment": "

Winner, 2024 Good Housekeeping Kids’ Book Award

If you’ve ever wondered why Squibs are never offered a place at Hogwarts, what happened when Vernon Dursley first met James Potter, or how Dumbledore and McGonagall formed their lifelong friendship, From the Wizarding Archive should go straight to the top of your wish list. Containing 80 articles written by J.K. Rowling for the original Pottermore website and available for the first time in audio, exclusively from Audible, Harry Potter fans everywhere are in for a treat.

If there’s one thing Harry Potter fans have in common (apart from impeccably good taste), it’s questions… so many questions. From the simple details that perplex us all – was Professor Umbridge always that awful? Why don’t wizards just use phones? – to the personal details that bring us closer to J.K. Rowling’s writing process – from her least favourite school subject and its impact on Professor Snape’s career, to the personal significance of King’s Cross and why it’s always where the Hogwarts Express departs – this is a veritable treasure trove of answers.

Narrated by Evanna Lynch, Hugh Quarshie, Finlay Robertson and Lara Sawalha, and with an exclusive foreword by Evanna Lynch, this spellbinding audiobook is essential listening for any Harry Potter aficionado.

The articles in From the Wizarding Archive were originally written by J.K. Rowling for pottermore.com and remain free to read in English on wizardingworld.com. 35 of the 80 articles were originally published across three Pottermore Presents eBooks. This is the first time any of the content has been available in audio format.

Theme music composed by Sara Carvalho.

Available in Dolby Atmos on Audible.

Just searching for mime didn't show any results. I don't know if it's listed under another name. But I guess the mimetype is the same as aac, so it's basically impossible for the app to request a transcode.

The app should use the native players, so if they can't play the files, it's probably a hardware incompatibility, but not sure here. (And because it does not fail, just not sound it also does not tries to transcode)

Yeah, just from my linux client box, it recognizes the mime type of the file as video/mp4 so it would be identical.

On the client app on android, wouldn't it be possible to add a routine to check MediaCodec support for EAC3 and behave differently when it encounters such codecs such as using a bundled decoder? But that doesn't address the web playback.

These are only going to become more common as more and more productions use immersive audio.

Do the server logs say anything when you are trying to play the file? The client will try to direct play the file and if it fails the server will attempt to transcode. Codec, mine type, and container support is not well documented for most devices, or can have custom implementations so we rely on whether the client fails to play the file.

I think the problem being that the client never failed. Like iOS does not fail when playing MP3 in M4B. It just plays silence.
Sadly if it's that: this is nothing that abs can fix

This error message momentarily flash in Audiobookshelf android app

Support for AC3 on browsers seem very mixed, and it seems very dependent on hardware and software support for chromium-based browsers if enabled at all.

Here are some methods to check if the browser supports the ac-3 codec:
https://github.com/cjw1115/enable-chromium-ac3-ec3-system-decoding?tab=readme-ov-file#how-to-verify-if-ac3ec3-supported

That said I can confirm that on a Windows 10 system which has the DolbyDecMFT dll I can play an AC3 audiobook with:

• Edge 142.0.3595.53
• Chromium 115.0.5747.0 (special build with hardware AC3 support enabled, see the repo mentioned above)

However, on every other browser it fails with these errors:

Just looking at the dev tools, there is a first request to the play endpoint after which there is a fail message in the console [PlayerHandler] Audio player error switching to HLS stream.

This triggers a second request to the play endpoint this time with forceTranscode: true and the player manages to load but as fails again because the actual data in the .ts files is still AC-3.

Is audiobookshelf capable of transcoding to an other codec?

On a separate note, when I initially tested this I had a huge CPU spike when loading the album with separate M4A AC-3 files per chapter, not sure if this related tho.

I also have the same error as @layanto on android

Ignore, mixed up container and codec for a second here: It should transcode to HLS. I am not sure if it might not be possible to transcode from AC3 to HLS?

Yeah, according to the specification HLS supports AAC, MP3, AC-3 and EAC-3.

Maybe the client could specifically tell the server that it doesn't have support for AC-3 in which case the server could transcode to AAC or MP3. This would mean loosing fidelity, and it might be nice to inform the client that this is happening.

As for the android app I have no idea how it could be done, streaming would be fine but downloaded files would still be AC-3

I do not have an AC-3 file to test, but I assume it has something to do with the mimetype.
As far as I can tell, it only tries specified formats for aac, and probably EAC-3 is already treated as aac (not entirely sure). There is a property that forces the aac codec to be used, but it is never used anywhere. I could make a version where you can supply it as post data, but since I do not have any file for this, I cannot test it and can only assume. But I guess this is the error.

That's the problem with proprietary formats ig

@TheStaticTurtle #4808

And thanks!

Awesome, I spun up a dev environment with this PR and did a test with the full audiobook, and I can confirm that it now works on multiple browsers.

However the android app still fails for both streaming and downloaded media.

Downloaded media still fails with the same error as @layanto probably because local media isn't transcoded before sending to the client so any codec issue would still be present?

Streaming on the other hand shows a generic Playback Failed: Source error message and nothing more in the app logs. I do see the Forcing AAC for tracks with mime type audio/mp4 and codec eac3 in the server logs. Looking at the logcat via adb, I can see the expected lines:

E PlayerListener: onPlayerError MediaCodecAudioRenderer error, index=1, format=Format(1, null, null, audio/eac3-joc, null, 768000, und, [-1, -1, -1.0], [6, 48000]), format_supported=YES
D PlayerNotificationServ: Fallback to transcode PlayItemRequestPayload(mediaPlayer=exo-player, forceDirectPlay=false, forceTranscode=true, deviceInfo=DeviceInfo(deviceId=xxxxxxxxxxxxxxxxxxxxxxxxx, manufacturer=Google, model=Pixel 9 Pro, sdkVersion=36, clientVersion=0.10.1-beta)).mediaPlayer

But a few lines later i'm greeted by:

E ExoPlayerImplInternal: Playback error
E ExoPlayerImplInternal:   com.google.android.exoplayer2.ExoPlaybackException: Source error
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:644)
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:616)
E ExoPlayerImplInternal:       at android.os.Handler.dispatchMessage(Handler.java:106)
E ExoPlayerImplInternal:       at android.os.Looper.loopOnce(Looper.java:248)
E ExoPlayerImplInternal:       at android.os.Looper.loop(Looper.java:338)
E ExoPlayerImplInternal:       at android.os.HandlerThread.run(HandlerThread.java:85)
E ExoPlayerImplInternal:   Caused by: com.google.android.exoplayer2.upstream.HttpDataSource$InvalidResponseCodeException: Response code: 404
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.upstream.DefaultHttpDataSource.open(DefaultHttpDataSource.java:413)
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.upstream.StatsDataSource.open(StatsDataSource.java:84)
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.source.hls.HlsMediaChunk.prepareExtraction(HlsMediaChunk.java:495)
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.source.hls.HlsMediaChunk.feedDataToExtractor(HlsMediaChunk.java:468)
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.source.hls.HlsMediaChunk.loadMedia(HlsMediaChunk.java:437)
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.source.hls.HlsMediaChunk.load(HlsMediaChunk.java:394)
E ExoPlayerImplInternal:       at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:412)
E ExoPlayerImplInternal:       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1156)
E ExoPlayerImplInternal:       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:651)
E ExoPlayerImplInternal:       at java.lang.Thread.run(Thread.java:1119)
E PlayerListener: onPlayerError Source error

(Response code: 404 being the issue)

Which I also noticed on the browser. Sometimes the transcode isn't ready yet and it takes quite a while for the first .ts file to exist. However unlike the app, the browser client re-tries the same segment up to 8 times ([HLS] Server 404 for fragment retry 2 of 8) before fully error-ing out which usually is enough time for the transcode to catch up.

This is probably worsened by the very slow dev machine I used for this test but I guess the android app doesn't have that retry mechanism.

Downloaded media still fails with the same error as @layanto probably because local media isn't transcoded before sending to the client so any codec issue would still be present?

Yes. It never has and probably never will be (but I am not a maintainer).

This is probably worsened by the very slow dev machine I used for this test but I guess the android app doesn't have that retry mechanism.

HLS always fails after a few seconds, then no transcoding would work, so there should be an retry (but I don't do dev on the official apps). One idea I have is to edit the duration. I don't know what it does, I just noticed it's higher for AAC. In the file I edited, there's a part of the code with 500 : 3000 (could be the other way around. Memory). For testing, please try increasing the 500 to 3000 or higher. I'm not completely sure what it does, and I'm on mobile so I can't test it, but maybe it's the time the client has to fetch.

@nichwall does dev on the offical app, maybe he can say more about retrying

HLS always fails after a few seconds, then no transcoding would work, so there should be an retry (but I don't do dev on the official apps). One idea I have is to edit the duration. I don't know what it does, I just noticed it's higher for AAC. In the file I edited, there's a part of the code with 500 : 3000 (could be the other way around. Memory). For testing, please try increasing the 500 to 3000 or higher. I'm not completely sure what it does, and I'm on mobile so I can't test it, but maybe it's the time the client has to fetch.

It looks like this code is only triggered if the transcode needs to be stopped and re-started, since AAC takes longer, the timeout that notifies that the stream is ready comes later. It doesn't affect the first transcode of a session.

What i'm noticing is that the player starts playing as soon as it gets the m3u8 playlist even if the segments don't exist. It appears that there already is a mechanism to tell the client when the HLS stream is ready (stream_ready event) but the app doesn't seem to use it.

Web is almost transparent (apart from the wait time) because of the re-try, however the android app just stops playing if it gets a 404 on a segment. If I restart everything (app & server & progress) I am able to start the stream, going backwards works but playback stops immediately due to a 404 if I jump into a non buffered section.

Yes. It never has and probably never will be (but I am not a maintainer).

That's probably fair, it seems that on android E-AC3 support is mixed, VLC plays it (of course lol) but not anything else I tested. There is apparently a plugin for MX-Player but it looks like they just embeded ffmpeg.

I does look like their is a way to support E-AC3 in the android media player per Dolby - https://professionalsupport.dolby.com/s/article/ExoPlayer-example-code-for-Dolby-Atmos-support?language=en_US

Even better, code examples - https://professionalsupport.dolby.com/s/article/Enabling-Dolby-Atmos-in-Android-Mobile-media-apps?language=en_US