It’s been a long time since a post but I wanted to highlight some of the new tooling that I’m learning while digging into the OmniSci platform – namely, extreme geospatial map rendering and analytics powered by server-side GPUs.
In this video I load a ~12M point dataset from geonames.org and explore it using both QGIS and OmniSci – just to get a feel for how easily it fits a normal kind of GIS exploration workflow.
Would love to hear how you partition your spatial data..
As data volumes grow, so does your need to understand how to partition your data. Until you understand this distributed storage concept, you will be unable to choose the best approach for the job. This post gives an introductory explanation of partitioning and you will see why it is integral to the Hadoop Distributed File System (HDFS) increasingly used in modern big data architectures….
Recently I started an Internet of Things series on my experiences installing, using and analysing data from a smart electrical meter. This included a BC Hydro smart meter, Eagle monitoring gateway from rainforest automation, and a cloud-based analytics service from Bidgely.
I’ve collated all the posts on the topic for you below. More will be added as I write them. Enjoy!
After a week of collecting smart meter readings, I’m now ready to show results in a cloud-based energy monitor system – Bidgely – complete with graphs showing readings, cost and machine learning results breaking down my usage by appliance.
This is part 4 of a series of posts about the Internet of Things applied to Home Energy Monitoring. I have a Smart Meter from BC Hydro, an Eagle energy monitor and various cloud apps helping me understand it all.
Working in the big data and analytics space, I’m always interested in parts of the Internet of Things (IoT) that will produce more data, require more backend systems, and help users/customers get on with their day better.
The past week has shown a few interesting announcements relating to Internet of Things topics. Here are just a few that jumped out to me, either because they inspired me or because I was left wondering what it would really mean.
TL;DR? Summary: While IBM is “getting started” (oops, I meant “getting serious”) and Facebook has big plans to “take over”, Amazon comes out with a consumer focused solution.
The Eagle energy monitor from Rainforest Automation is a very handy device. It reads the wireless signal from my electricity meter and makes it available through a web interface – both a graphical environment and a RESTful API. In this post we look at the standard graphical screens and the data download option. Next time we’ll look at the RESTful API for programmers to use.
This is part 3 of a series of posts about the Internet of Things applied to Home Energy Monitoring.
This definitely sounds promising, anyone have experience running a similar setup?
A small company, SpinRay Energy, has announced the production of a new UL listed, grid-tied solar power system that couldn’t be easier to install. Plug-in solar power equipment may be a game changer in the arena of small residential solar power.
I see two scenarios for those in rental environments – hanging something from windows or setting up on a porch. Ultimately I guess the efficacy will be limited by size availability and cost, but size issues could be minimised if it were possible to hang PV systems in some manner without needing to screw things down into exterior walls.
Just a thought – you tried anything like this? Share any links to your favourite portable, yet substantial, systems.
Energy monitoring isn’t only about knowing what’s happening right now, but also understanding what happened historically. Often that includes knowing not just what was happening but also when and why. Enter cloud services for energy monitoring. They range from simple charts and graphs to predicting your usage over time – essentially storing, analysing and enriching your raw data.
In this post I review two cloud services that I’ve tried so far and show you what you get from them. Continue reading IoT Day 2: Cloud Services for Energy Monitoring
In my next series of blog posts we explore an Internet of Things topic – Home energy monitoring – from a first person perspective. Join me as I install, use and hack a monitor (and related cloud services) in my new home.
This is part 1 of a series of posts about the Internet of Things applied to Home Energy Monitoring.
Between classic business transactions and social interactions and machine-generated observations, the digital data tap has been turned on and it will never be turned off. The flow of data is everlasting. Which is why you see a lot of things in the loop around real time frameworks and streaming frameworks. – Mike Hoskins, CTO Actian
From Mike Hoskins to Mike Richards (yes we can do that kind of leap in logic, it’s the weekend)…
Oh, Joel Miller, you just found the marble in the oatmeal! You’re a lucky, lucky, lucky little boy – because you know why? You get to drink from… the firehose! Okay, ready? Open wide! – Stanley Spadowski, UHF
Firehose of Terror
I think you get the picture – a potentially frightening picture for those unprepared to handle the torrent of data that is coming down the pipe. Unfortunately, for those who are unprepared, the disaster will not merely overwhelm them. Quite the contrary – I believe they will be consumed by irrelevancy.