When streaming response in Flask file unplayable

Question

I currently have a function that runs ffmpeg enconder on a flv stream from youtube.

def console(cmd, add_newlines=False):
    p = Popen(cmd, shell=True, stdout=PIPE)
    while True:
        data = p.stdout.readline()
        if add_newlines:
            data += str('\n')
        yield data

        p.poll()
        if isinstance(p.returncode, int):
            if p.returncode > 0:
                # return code was non zero, an error?
                print 'error:', p.returncode
            break

This works fine when I run the ffmpeg command and have it output to a file. The file is playable.

mp3 = console('ffmpeg -i "%s" -acodec libmp3lame -ar 44100 -f mp3 test.mp3' % video_url, add_newlines=True)

But when I have ffmpeg output to stdout via - instead of test.mp3, and stream that response. The file streams fine, is the correct size. But does not play correctly. Sounds chopy, and when I check the properties of the file it doesn't show the data of it as it does with test.mp3

@app.route('/test.mp3')
def generate_large_mp3(path):
    mp3 = console('ffmpeg -i "%s" -acodec libmp3lame -ar 44100 -f mp3 -' % video_url, add_newlines=True)
    return Response(stream_with_context(mp3), mimetype="audio/mpeg3",
                   headers={"Content-Disposition":
                                "attachment;filename=test.mp3"})

Is there something I am missing?

What is console function? Is it from a third-party library? — Aleksandr Kovalev
– Aleksandr Kovalev, Commented Dec 21, 2015 at 6:54
the console function is defined in the question, it is to use subprocess from python to run ffmpeg to do the conversion. — nadermx
– nadermx, Commented Dec 21, 2015 at 7:31
unrelated to you mp3 case: (1) .readline() returns a string with the newline already (unless the last line doesn't end with a newline). (2) str('\n') is pointless: '\n' by itself is already str object; unless there is from __future__ import unicode_literals on Python 2. (3) Don't call .poll() in a loop: here's a correct way to read text lines from a subprocess — jfs
– jfs, Commented Dec 21, 2015 at 21:39
add_newlines=True should corrupt your binary mp3 data unless mp3 allows to double all '\n' in the stream. — jfs
– jfs, Commented Dec 21, 2015 at 21:42

Miguel Grinberg · Accepted Answer · 2015-12-21 20:48:31Z

24

+100

You have a couple of issues. It seems you copied this console function from somewhere. The main problem is that this function is designed to work on text output. The command that you are running with ffmpeg will dump the binary raw mp3 data to stdout, so reading that output as if it was a text file with readlines() will not work. The concept of lines does not exist at all in the mp3 binary stream.

I have adapted your console() function to work on binary streams. Here is my version:

def console(cmd):
    p = Popen(cmd, shell=True, stdout=PIPE)
    while True:
        data = p.stdout.read()
        yield data

        p.poll()
        if isinstance(p.returncode, int):
            if p.returncode > 0:
                # return code was non zero, an error?
                print 'error:', p.returncode
            break

Note that I have removed all the references to newline handling, and also replaced readline() with read().

With this version I get the streaming to work. I hope this helps!

Edit: You could force the response to go out in chunks of a specified size by calling read(N) with N being the chunk size. For a given audio bitrate, you can calculate the size in bytes for a chunk of audio with a duration of say, 5 seconds, and then stream 5 second chunks.

edited Dec 21, 2015 at 20:48

answered Dec 21, 2015 at 8:08

Miguel Grinberg

68k14 gold badges142 silver badges160 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

nadermx Over a year ago

unfortunately this half works, the file is playable but It does what I currently have where it downloads the entire file before sending it to the user. What I'm trying to avoid here is when a large mp3 is put the client has to wait for the entire song to convert before sending it.

Miguel Grinberg Over a year ago

I think "streaming" in this context can be interpreted as two different things. On the server, streaming means that you can provide the response in chunks. That's the part I answered here, and I believe that is the "half" that works. The part that remains, is to make the browser instantiate an audio player and start playing while the response is still streaming. For that, you need to embed an audio player, I think. See stackoverflow.com/questions/21450930/… for some ideas.

jfs Over a year ago

(1) while True: data = p.stdout.read() is wrong. .read() won't return until EOF. OP could use os.read(p.stdout.fileno(), 8192) instead (os.read() returns as soon as there is any data available). (2) don't call p.poll(), use if not data: break instead.

Miguel Grinberg Over a year ago

@nadermx awesome! But do note the comment by @J.F.Sebastian regarding the potential for the read() call to block. To avoid that, you can do what he says, or else use a block size that is a multiple of the mp3 frame size for your chosen bitrate.

Denis Barmenkov 7 hours ago

if p.returncode != 0: ...

jfs · Accepted Answer · 2015-12-21 22:31:12Z

To stream mp3 content generated by a subprocess using flask:

#!/usr/bin/env python
import os
from functools import partial
from subprocess import Popen, PIPE

from flask import Flask, Response  # $ pip install flask

mp3file = 'test.mp3'
app = Flask(__name__)


@app.route('/')
def index():
    return """<!doctype html>
<title>Play {mp3file}</title>
<audio controls autoplay >
    <source src="{mp3file}" type="audio/mp3" >
    Your browser does not support this audio format.
</audio>""".format(mp3file=mp3file)


@app.route('/' + mp3file)
def stream():
    process = Popen(['cat', mp3file], stdout=PIPE, bufsize=-1)
    read_chunk = partial(os.read, process.stdout.fileno(), 1024)
    return Response(iter(read_chunk, b''), mimetype='audio/mp3')

if __name__ == "__main__":
    app.run()

Replace ['cat', mp3file] with your ffmpeg command that writes mp3 content to its stdout.

Collectives™ on Stack Overflow

When streaming response in Flask file unplayable

2 Answers 2

5 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related