The Device Chronicle interviewed Luke Murray, CTO of XY Sense, on how AI-enabled IoT sensors are used to improve office space utilization and the importance of OTA software updates in keeping the sensors robust and secure, and the AI applications updated.
Luke has extensive experience in commercial real estate. One of the most common problems he observed before he founded XY Sense was that tenants of large office spaces (a large bank with 20,000 employees), didn’t have data and insights on how that space was being used. Yet they were making non-data driven decisions on space utilisation involving tens of millions to hundreds of millions of dollars.
Use case for IoT sensors
Luke explains that a piece of property is typically an enterprise’s second or third largest expense after salaries. There was a real gap in the market to provide these organisations with the data so they could make better strategic decisions. If they keep getting requests but that there’s no space and they keep having to go out and sign multi-million dollar leases, but every time they do a one-off survey, it comes back that 40% is not used.
They want to make better decisions so there’s that strategic piece to understand space utilization over a long time and to understand how to grow more efficiently into a space. When is the right time to grow into new space strategically, and an organisation might typically have goals to consolidate into a central sort of headquarters or distribute out and understand what types of spaces are being used. All this feeds into having a better experience and building a better experience for the people that use the space leading to improved productivity.
With offices gradually opening up after an historic 12 months or more of ‘working from home’ due to the pandemic, we can see that the way we work may have changed forever. Businesses around the world are adopting hybrid workplace models offering greater flexibility for workers who can choose when they work from the office. Achieving real-time anonymous utilization of space can feed into workplace experience tools, so people can find the type of space they need to sort of collaborate with their team or alternatively, find a quiet place or a meeting room to get work done when they need to.
Impact of Covid on demand for IoT sensors
Luke explains that the Pandemic created a higher demand for data as companies grappled with employees partially working from home on the one hand, and having to reduce their office rental portfolio on the other. But they don’t want to jump into rash decisions and then have to acquire more portfolios if they make the wrong choice. So they really want to make that decision based on data. And having that sort of data collected over the long term.
XY Sense IoT sensor solution
XY Sense has one main offering with other developments on their roadmap. Their product is a ceiling mounted sensor which provides real time data about how an office space is being used. It is computer vision based. Everything is processed on the sensor though and the only thing that leaves the sensor is anonymous X Y coordinates representative of a person on an office floor plan
Engineering a privacy-preserving design was critical for XY Sense whose customers are large corporate employers who take the information security and employee privacy protections of their workplaces seriously. XY Sense is designed so that just the XY coordinates leave the device and each unit covers about 95 square meters of open space. Being computer vision-based, this obviously means ‘line of sight’ is a requirement so we need to work around walls and obstacles. That being said, XY Sense has one of the largest open space coverage for this type of utilization sensor. Proprietary AI is run on the edge of the device itself to allow for real time processing. The core solution is cabled and customers typically install the sensors for the lifetime of their lease. In this scenario, total cost to ownership is far lower when the sensors are cabled in. Usually, the office buildings are five or six star rated green energy buildings and the sensors can utilise this. It’s also possible to daisy chain the sensors together to really minimize the installation costs.
Hardware and software in this IoT sensor
Under the hood of the XY Sense is a system on a chip from Xilinx with two ARM core processors running on it, with an FPGA (Field Programmable Gate Arrays). The sensor works with a custom neural network that has been trained, and Luke says, continues to improve over time. “We have a pipeline where we, we actually have a FPGA in this thing which we use as the neural network accelerator. So it’s obviously got some arm calls. It is running the general operating system, preparing data and passing it into the FPGA. We take machine learning models, and train and optimize them offline. We have done a lot to get something that typically requires a large GPU with gigabytes of memory to run on a tiny and low powered embedded device. We’re collecting this data every two seconds, which really enables our toolset and integrations with building management solutions and booking solutions.”
Object detection with IoT sensors done anonymously
Today, XY Sense is designed to anonymously detect people as objects. Luke explains “It is object detection where we locate where that anonymous person is on a floor plan typically seen within 30cm of locational accuracy. Typically in the use case of office space utilisation, you are looking for the 5 desks that are never being used so you can repurpose them. If you can pinpoint the employees that are within a 30 cm radius in the space then you can better understand how individual spaces can be utilised and what types of spaces will work best – whether these should be collaborative spaces, or meeting rooms.”
Data driven insights from IoT sensors
Luke believes that it’s really helpful to have insights from the data as more and more companies move to activity-based working or flexible working. “It’s really helping having this data for these companies to help office space planners and managers communicate plans and changes to teams. A typical scenario: Based on the usage over the last three months, we can help companies project when and how they may run out of space in this flexible environment because of the way their employees work. Planners may be worried that too many employees come in on the same day but they get insights from the data that shows them trends across days of the week and they cna use that information to match demand and to help transition people into a new way of working. The real estate teams at these large organizations measure continuously as they make changes and they can reflect on how these changes affected total utilization and real estate costs. They can also survey and understand how it’s affecting employees in the office space and observe utilization over time.
IoT sensors and OTA software updates
Luke explains that OTA software updates were very important to XY Sense from day one. Luke says that they quickly identified that they were not just going to develop their algorithms and push them out one time to thousands of sensors and leave it at that. Instead, he explains that they look to continuously improve the performance of the algorithms, as well as adding new features to the sensor through extensibility. “It really opens up as we can train the sensor to do new things. Enterprise customers are also not looking to invest in hardware that is obsolete in 6 or 12 months time. The field of AI and computer vision is moving fast. Customers want to know that they’re getting the sensor and it’s going to continue to be improved. We perform full system updates for the embedded Linux operating system and security patch updates. There are also 2 applications that run on the hardware and need to be updated from our neural network. We push new models of that. We have to update the controller software and the application that is maintaining the IoT MQTT connection back to the cloud for configuration tasks.”
We wish Luke and the team at XY Sense well as they advance the use of AI-powered IoT sensors.