Camera Working Group – March 2017 Update

The Camera Working Group (WG) is focused on platform-independent developer APIs for cameras, and for remote control of camera and gimbal. APIs under investigation currently include:

• Remote Camera and Gimbal Control APIs
• Video Streaming APIs
• C/C++ developer camera APIs (FPV, Snapshot, Recording, Optic Flow, and Stereo)

Check out our accomplishments and challenges for March 2017 below, along with our plans for April.

Accomplishments

• MAVLINK Camera and Gimbal API improvements were merged (Julian Oes):
  • https://github.com/mavlink/mavlink/pull/635
  • https://github.com/mavlink/mavlink/pull/641
• Optic Flow support for Bebop was merged:
  • https://github.com/PX4/Firmware/pull/6376
• Qualcomm Technologies Inc provided links to the platform independent Camera APIs used in Snapdragon Flight, for consideration and comment from TSC and Camera WG (the APIs are already public):
  • https://source.codeaurora.org/quic/le/platform/hardware/qcom/camera/tree…
  • https://source.codeaurora.org/quic/le/platform/hardware/qcom/camera/tree…
  • There is a sample app using the API on Github:
    • https://github.com/PX4/snap_cam/

Challenges

• Waiting for feedback on the proposed C++ Camera API.
• HW Availability for Snapdragon Flight Stereo Camera.
• Issues with PX4 Optic Flow on Snapdragon Flight
• uORB issues blocking (should now be resolved).
• Intel Aero and Snapdragon flight Optic Flow integration with PX4 currently not working.
• Code availability for Intel Aero video API.

Plan for next month

• Continue working on Optic Flow support in PX4.
• Continue working on VISLAM support for PX4.
• Review feedback on Camera API.

Additional Information for working with Camera and Video

• Intel’s work on video streaming with GStreamer:
  • Camera Streaming custom MAVLink message (discuss.ardupilot.org)
• Snapdragon Flight supports video streaming via live555:
  • ​Snapdragon_Linux_Camera_Interface_Specification.pdf
  • Note that the use of GStreamer or live555 should not be exposed to the final video streaming API. The API should be agnostic to the underlying implementation.
• Camera information for Snapdragon Flight is in Chapter 4 of: ​
  • 80-P8822-1_A_Qualcomm_Snapdragon_Flight_User_Guide.pdf
• Optical Flow on the Snapdragon Flight:
  • Optical Flow on the Snapdragon Flight (PX4 Dev Guide)
  • https://github.com/PX4/snap_cam
• Information on the Intel Aero Vision Kit is at:
  • https://software.intel.com/en-us/aero/vision-kit