Android Media Devices¶

Example of Android application managing media devices¶

This example can be used as streamer allowing to select source camera and microphone and specify parameters for the published video: FPS (Frames Per Second) and resolution (width, height).

Two videos are played

left - video from the camera
right - the published video stream as received from the server

Also the input field for streaming parameters are shown

Switching video renderer and camera:

Analyzing the example code¶

To analyze the code, let's take class MediaDevicesActivity.java of the media-devices example, which can be downloaded with the build 1.1.0.69.

1. Initialization of the API¶

Flashphoner.init() code

For initialization, Сontext object is passed to the init() method.

Flashphoner.init(this);

2. List available media devices¶

Flashphoner.getMediaDevices(), MediaDeviceList.getAudioList(), MediaDeviceList.getVideoList() code

mMicSpinner = (Spinner) findViewById(R.id.microphone);
ArrayAdapter<MediaDevice> micAdapter = new ArrayAdapter<>(
        this,
        android.R.layout.simple_spinner_item,
        Flashphoner.getMediaDevices().getAudioList()
);
micAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
mMicSpinner.setAdapter(micAdapter);
...
mCameraSpinner = (Spinner) findViewById(R.id.camera);
ArrayAdapter<MediaDevice> camAdapter = new ArrayAdapter<>(
        this,
        android.R.layout.simple_spinner_item,
        Flashphoner.getMediaDevices().getVideoList()
);
camAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
mCameraSpinner.setAdapter(camAdapter);

3. Video render management¶

FPSurfaceViewRenderer.setMirror() code

When a video is shown, an image is displayed to FPSurfaceViewRenderer objects:

localRender to display video from camera
remoteRender to display stream published preview
newSurfaceRenderer to demonstrate renderer switching

For those objects, screen position, scaling type and mirroring should be set.

By default, mirror view is set to display video from camera by setMirror(true) method invokation. To display stream published preview and renderer switching object, mirroring is switched off by setMirror(false):

remoteRenderLayout.setPosition(0, 0, 100, 100);
remoteRender.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT);
remoteRender.setMirror(false);
remoteRender.requestLayout();

localRenderLayout.setPosition(0, 0, 100, 100);
localRender.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT);
localRender.setMirror(true);
localRender.requestLayout();

switchRenderLayout.setPosition(0, 0, 100, 100);
newSurfaceRenderer.setZOrderMediaOverlay(true);
newSurfaceRenderer.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT);
newSurfaceRenderer.setMirror(true);
newSurfaceRenderer.requestLayout();

In this case, when you choose front camera, the image displayed from camera looks normally but is published as mirrored. When you choose back camera, image from camera looks mirrored but is publihed in normal orientation (see application screenshots above).

4. Getting audio and video constraints set by user¶

AudioConstraints, VideoConstraints code

@NonNull
private Constraints getConstraints() {
    AudioConstraints audioConstraints = null;
    if (mSendAudio.isChecked()) {
        audioConstraints = new AudioConstraints();
        if (mUseFEC.isChecked()) {
            audioConstraints.setUseFEC(true);
        }
        if (mUseStereo.isChecked()) {
            audioConstraints.setUseStereo(true);
        }
        if (!mDefaultPublishAudioBitrate.isChecked() && mDefaultPublishAudioBitrate.getText().length() > 0) {
            audioConstraints.setBitrate(Integer.parseInt(mPublishAudioBitrate.getText().toString()));
        }
    }
    VideoConstraints videoConstraints = null;
    if (mSendVideo.isChecked()) {
        videoConstraints = new VideoConstraints();
        videoConstraints.setCameraId(((MediaDevice) mCameraSpinner.getSelectedItem()).getId());
        if (mCameraFPS.getText().length() > 0) {
            videoConstraints.setVideoFps(Integer.parseInt(mCameraFPS.getText().toString()));
        }
        if (mWidth.getText().length() > 0 && mHeight.getText().length() > 0) {
            videoConstraints.setResolution(Integer.parseInt(mWidth.getText().toString()),
                    Integer.parseInt(mHeight.getText().toString()));
        }
        if (!mDefaultPublishVideoBitrate.isChecked()) {
            setVideoBitrate(videoConstraints);
        }
    }
    return new Constraints(audioConstraints, videoConstraints);
}

5. Local camera and microphone testing¶

Flashphoner.getLocalMediaAccess() code

The following parameters are passed:

audio and video constarints set by user
local object SurfaceViewRenderer localRenderer to display image from camera

case TEST_REQUEST_CODE: {
    if (grantResults.length == 0 ||
          grantResults[0] != PackageManager.PERMISSION_GRANTED ||
          grantResults[1] != PackageManager.PERMISSION_GRANTED) {
        Log.i(TAG, "Permission has been denied by user");
    } else {
        Flashphoner.getLocalMediaAccess(getConstraints(), localRender);
        mTestButton.setText(R.string.action_release);
        mTestButton.setTag(R.string.action_release);
        mStartButton.setEnabled(false);
        soundMeter = new SoundMeter();
        soundMeter.start();
        ...
        Log.i(TAG, "Permission has been granted by user");
    }
break;

6. Session creation¶

Flashphoner.createSession() code

SessionOptions object with the following parameters is passed to the createSession() method:

URL of WCS server

SessionOptions sessionOptions = new SessionOptions(url);

/**
  * Session for connection to WCS server is created with method createSession().
  */
session = Flashphoner.createSession(sessionOptions);

7. Connection to the server¶

Session.connect() code

session.connect(new Connection(), getBasicAuthHeader(url));

8. Receiving the event confirming successful connection¶

Session.onConnected() code

@Override
public void onConnected(final Connection connection) {
    runOnUiThread(new Runnable() {
        @Override
        public void run() {
            mStatusView.setText(connection.getStatus());
            mPublishButton.setEnabled(true);
            mPlayButton.setEnabled(true);
            //onStarted();
            MediaDevicesActivity.this.onConnected();
        }
    });
}

9. Video stream creation to publish¶

Session.createStream() code

publishStream = session.createStream(streamOptions);
...

ActivityCompat.requestPermissions(MediaDevicesActivity.this,
         new String[]{Manifest.permission.RECORD_AUDIO, Manifest.permission.CAMERA},
         PUBLISH_REQUEST_CODE);

10. Video stream publishing¶

Stream.publish() code

case PUBLISH_REQUEST_CODE: {
    if (grantResults.length == 0 ||
           grantResults[0] != PackageManager.PERMISSION_GRANTED ||
           grantResults[1] != PackageManager.PERMISSION_GRANTED) {
        mStartButton.setEnabled(false);
        mTestButton.setEnabled(false);
        session.disconnect();
        Log.i(TAG, "Permission has been denied by user");
    } else {
        /**
          * Method Stream.publish() is called to publish stream.
          */
        publishStream.publish();
        Log.i(TAG, "Permission has been granted by user");
    }
    break;
}

11. Receiving the event confirming successful stream publishing¶

StreamStatusEvent.PUBLISHING code

publishStream.on(new StreamEventHandler() {
    @Override
    public void onStreamStatus(final Stream stream, final StreamStatus streamStatus) {
        runOnUiThread(new Runnable() {
            @Override
            public void run() {
                if (StreamStatus.PUBLISHING.equals(streamStatus)) {
                    onPublished();
                } else {
                    onUnpublished();
                    Log.e(TAG, "Can not publish stream " + stream.getName() + " " + streamStatus);
                }
                mStatusView.setText(streamStatus.toString());
            }
        });
    }

    @Override
    public void onStreamEvent(StreamEvent streamEvent) {

    }
});

12. Video stream creation to play¶

Session.createStream() code

playStream = session.createStream(streamOptions);

13. Video stream playing¶

Stream.play() code

playStream.play();

14. Receiving the event confirming successful stream publishing¶

StreamStatusEvent.PLAYING code

playStream.on(new StreamEventHandler() {
    @Override
    public void onStreamStatus(final Stream stream, final StreamStatus streamStatus) {
        runOnUiThread(new Runnable() {
            @Override
            public void run() {
                if (!StreamStatus.PLAYING.equals(streamStatus)) {
                    onStoppedPlay();
                    Log.e(TAG, "Can not play stream " + stream.getName() + " " + streamStatus);
                } else {
                    onPlayed(stream);
                    Flashphoner.setVolume(mPlayVolume.getProgress());
                }
                mStatusView.setText(streamStatus.toString());
            }
        });
    }

    @Override
    public void onStreamEvent(StreamEvent streamEvent) {
        runOnUiThread(new Runnable() {
            @Override
            public void run() {
                if (streamEvent.getPayload() != null) {
                    mMutedName.setText(getString(R.string.muted_name) + streamEvent.getPayload().get("streamName"));
                }
                switch (streamEvent.getType()) {
                    case audioMuted:
                        mAudioMuteStatus.setText(getString(R.string.audio_mute_status) + "true");
                        break;
                    case audioUnmuted:
                        mAudioMuteStatus.setText(getString(R.string.audio_mute_status) + "false");
                        break;
                    case videoMuted:
                        mVideoMuteStatus.setText(getString(R.string.video_mute_status) + "true");
                        break;
                    case videoUnmuted:
                        mVideoMuteStatus.setText(getString(R.string.video_mute_status) + "false");
                }
            }
        });
    }
});

15. Switching camera to the next one while publishing stream¶

Stream.switchCamera() code

private void switchToCamera() {
    if (publishStream != null) {
        turnOffFlashlight();
        muteButton();
        publishStream.switchCamera(new CameraSwitchHandler() {
            @Override
            public void onCameraSwitchDone(boolean var1) {
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        if (mConnectButton.getTag() == null || Integer.valueOf(R.string.action_disconnect).equals(mConnectButton.getTag())) {
                            onConnected();
                            onPublished();
                        }
                    }
                });

            }

            @Override
            public void onCameraSwitchError(String var1) {
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        onConnected();
                        onPublished();
                    }
                });
            }
        });
    }
}

16. Switching to the certain camera by name while publishing stream¶

Stream.switchCamera() code

private void switchToCamera(String name) {
    if (publishStream != null) {
        turnOffFlashlight();
        muteButton();
        publishStream.switchCamera(new CameraSwitchHandler() {
            @Override
            public void onCameraSwitchDone(boolean var1) {
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        if (mConnectButton.getTag() == null || Integer.valueOf(R.string.action_disconnect).equals(mConnectButton.getTag())) {
                            onConnected();
                            onPublished();
                        }
                    }
                });
            }

            @Override
            public void onCameraSwitchError(String var1) {
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        onConnected();
                        onPublished();
                    }
                });
            }
        }, name);
    }
}

17. Switching renderer object while publishing stream¶

Stream.switchRenderer() code

private void switchRender() {
    if (spinner.getSelectedItemId() == 0) {
        if (isSwitchLocalRenderer) {
            publishStream.switchRenderer(localRender);
            isSwitchLocalRenderer = false;
        } else {
            publishStream.switchRenderer(newSurfaceRenderer);
            isSwitchLocalRenderer = true;
        }
    } else {
        if (isSwitchRemoteRenderer) {
            playStream.switchRenderer(remoteRender);
            isSwitchRemoteRenderer = false;
        } else {
            playStream.switchRenderer(newSurfaceRenderer);
            isSwitchRemoteRenderer = true;
        }
    }
}

18. Sound level management using hardware buttons¶

Flashphoner.setVolume() code

@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
    int currentVolume = Flashphoner.getVolume();
    switch (keyCode) {
        case KeyEvent.KEYCODE_VOLUME_DOWN:
            if (currentVolume == 1) {
                Flashphoner.setVolume(0);
            }
            mPlayVolume.setProgress(currentVolume-1);
            break;
        case KeyEvent.KEYCODE_VOLUME_UP:
            if (currentVolume == 0) {
                Flashphoner.setVolume(1);
            }
            mPlayVolume.setProgress(currentVolume+1);
            break;
    }
    return super.onKeyDown(keyCode, event);
}

19. Device speakerphone usage¶

Flashphoner.getAudioManager().setUseSpeakerPhone() code

mAudioOutput.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {
    @Override
    public void onItemSelected(AdapterView<?> adapterView, View view, int i, long l) {
        String audioType = (String) adapterView.getItemAtPosition(i);
        switch (audioType) {
            case "speakerphone":
                Flashphoner.getAudioManager().setUseSpeakerPhone(true);
                break;
            case "phone":
                Flashphoner.getAudioManager().setUseBluetoothSco(false);
                Flashphoner.getAudioManager().setUseSpeakerPhone(false);
                break;
            case "bluetooth":
                Flashphoner.getAudioManager().setUseBluetoothSco(true);
                break;
        }
    }

    @Override
    public void onNothingSelected(AdapterView<?> adapterView) {

    }
});

20. Session disconnection¶

Session.disconnect() code

session.disconnect();

21. Receiving the event confirming successful disconnection¶

Session.onDisconnection() code

@Override
public void onDisconnection(final Connection connection) {
    runOnUiThread(new Runnable() {
        @Override
        public void run() {
            mStatusView.setText(connection.getStatus());
            ...
            mPublishButton.setEnabled(false);
            mPlayButton.setEnabled(false);
            //onStopped();
            MediaDevicesActivity.this.onDisconnected();
        }
    });
}