Innovative 3D camera technology wins OKdo engineering challenge

The ROCK Engineering Challenge invited engineers, entrepreneurs, inventors, and developers to submit ideas for parts, products, or systems that take advantage of ROCK single-board computers. 

We received more than 150 entries from across the globe, with ideas ranging from search and rescue drones to open-source care devices. 

The Challenge jury carefully selected the top ten projectsbased on their impact on the industry, their scalability, and their feasibility. 

The overall winner is Eric Schleicher from the company Constructive Realities. Constructive Realities specializes in computer vision and 3d reconstruction applications. Constructive Realities wins a $50,000 partnership with OKdo to scale their ideas. 

The Project: 

The Californian-based Constructive Realities are working to bring to market a product line of Time of Flight (ToF) 3D cameras in a compact "compute-included" form factor. The technology is targeted to the consumer (AR/VR & 3d Scanning/IoT), maker, and robotics and industrial product markets.

Generally available ToF sensors are either very low resolution (320x240 or less) or require a powerful host system on which to perform raw data processing. Their system offers the distinct advantage of running a depth data pipeline NPUs and GPUs commonly found on small ARM SBCs and SoMs like the raspberry pi rockchip 3566/3588.

“We will be offering a depth (only) model alongside an upgraded version that combines the 3D camera with Color fusion (RGB-D) devices,” says Schleicher . “We have a python based API that will allow users to process the depth data on device and enable various IoT pieces of functionality or a host side library for the RGB fusion devices that allows for 3D reconstruction use cases.”

“We have been producing a version of this system for research and development purposes for robotics and smart infrastructure companies based on the myriad X VPU. This has enabled us to develop the software to support these devices and data streams and we offer these products on B2B basis. The lack of userspace and support for the myriad X makes it difficult to deliver to the general public and leaves no room or possibility on device for custom logic for particular use cases. Linux based SoM/SBCs provide for that extensibility and community at a similar price point in a way that we believe can seriously impact all levels of the vision and robotics community,” Schleicher further explains. 

How do Constructive Realities take advantage of the features of that ROCK single-board computer? 

Eric Schleicher of Constructive Realities, in his company's OKdo Engineering Challenge submission, highlighted their preference for the CM3/i, due to its efficient compatibility with their existing pipeline, which runs at over 30 fps on the RK3566. The device's form factor is ideal for leveraging the CM3's advanced capabilities, including the Pi-friendly color fusion imager interposer. 

While the RPI CM4 can handle the CSI2, basic processing, and USB bandwidth necessary for their depth-only use case, their RGB-D device requires USB 3.0 for transferring processed streams up to 4k RGB data. They aim to standardize around the Radxa CM3/CM3i for both depth-only and color-fusion use cases in collaboration with Okdo/Radxa, favoring its simplicity and consistency over using different devices. The potential to adapt the CM3i for industrial applications also holds significant appeal for them and their customers.

What’s next for Constructive Realities?

Constructive Realities has successfully tested its pipeline on a wide range of platforms, from Pi zeros to high-performance x64 platforms and GPUs. The company sees immense scalability and opportunity for their products, given the relevance of their imaging technology in the fields of robotics and smart devices. The goal now is to find a computing module partner that can help deliver their 3D capabilities in a compact, cost-effective, and user-friendly device. If the Radxa CM3/i is the platform that makes this feasible, they predict a unit volume of over 10,000 units annually within 18 months of release.

The company is moving at a rapid pace, with PCBs scheduled for assembly within the next two weeks. This will allow them to begin testing with CSI connectivity on their 3566 comparison test systems using existing COTS carrier boards. Extended testing of these DVT camera interposers will take place in May. As they validate these Amlogic/Pi4/3566 SoMs, they plan to finalize the decision on which modules to use for their initial offering and confirm their production intent baseboard design.

In June, they will deploy evaluation units to their pre-arranged development partners and support a fixed number of trials with an open call for project submissions. Marketing tests are planned for the summer, with a view to launching a crowdfunding campaign where they can demonstrate fully working units in a variety of compelling applications. They anticipate this will happen in late July or August as they finalize what they can offer to the public without over-promising.

The company seeks to avoid a dual-make solution, either with a mix of SBC/CM4 footprints or a mix between SoM vendors. Their research has highlighted the Radxa CM3/i as a potential favorite, given its capacity to support both depth-only and color-fusion with a single make/format and its USB3/DRD compatibility. Their primary requirement is to gain access to the CM3i and the necessary support to validate USB3 DRD functionality. If the CM3/CM3i cannot be used as planned, they may delay the depth-only versions of the system and select another SoM/SBC, developing a custom solution for USB3 client/DRD.

While they already have running proof of concepts with the same processor, Constructive Realities lacks a hardware partner that can ensure the availability of compute modules. They have approximately 1500 ToF sensors on hand that they expect to sell in 2023. Therefore, understanding the timing, availability, and cost of these SoMs is crucial to their operation and will significantly influence their final selection.

Their team is equipped with resources for board layout and design, but their primary expertise lies in sensor fusion, software, and product design. They hope that external participation and access to RK/SoC domain experts will expedite the development of their production baseboard. This collaboration could also help in understanding thermal design considerations, given that their system is expected to run reliably 24/7, and the sensor and GPU will be used almost continuously.

Read more about the company here.

Wevolver Community Vote winner

The Wevolver Community Vote winner was selected by the engineering community through a public poll. The winner of the Community Vote wins the opportunity to pitch their ideas to the senior management of OKdo and its partners for a chance at establishing partnerships.

The Wevolver community winner of the OKdo charge is Marcel Ochsendorf with a thermal imaging medical device. 

This project introduces a product designed to aid medical facilities by using a thermal imaging camera to non-invasively record a patient's status. The system can determine breathing status, frequency, body temperature, and movements like falls, alerting medical staff when necessary.

The use of a thermal imaging camera and local image recognition system ensures complete anonymization of patients, as no sensitive data leaves the room. Feedback from preliminary discussions with hospitals has been largely positive, appreciating the system's data protection and functionality. While the focus is primarily on breathing and condition detection, additional sensors can be integrated for further situational awareness. A trial using a basic thermal imaging camera and image recognition software has confirmed the components' suitability.

The Rock board, with its dedicated GPU, provides a significant advantage for this project, as it ensures local processing for data protection - a requirement often unmet by other systems. Initial tests on a standard computer have demonstrated that the Rock Single Board Computer (SBC) is the optimal choice for this task, offering not only the necessary performance but also ample resources for future upgrades.

We'll feature a deeper interview with Marcel Ochsendorf and his technology soon. 

Read more about the ROCK series of single-board computers here.