Browse Source

Remove deprecated software

Delete the old `meteod` system, as meteo will be based on `influxdb`
only in the future.
pull/15/head
Felix 5 months ago
parent
commit
7d049f20b8
Signed by: phoenix
GPG Key ID: 6E77A590E3F6D71C
  1. 2
      .gitignore
  2. 35
      Makefile
  3. 66
      README.md
  4. 2
      cmd/gateway/.gitignore
  5. 5
      cmd/gateway/Makefile
  6. 5
      cmd/gateway/README.md
  7. 184
      cmd/gateway/gateway.go
  8. 0
      cmd/meteo-influx-gateway/influxdb.go
  9. 0
      cmd/meteo-influx-gateway/meteo-mqtt-influxdb.go
  10. 0
      cmd/meteo-influx-gateway/mqtt.go
  11. 297
      cmd/meteo/meteo.go
  12. 1482
      cmd/meteod/meteod.go
  13. 6
      go.mod
  14. 8
      go.sum
  15. 0
      integrators/home-assistant/README.md
  16. 0
      integrators/home-assistant/meteo/__init__.py
  17. 0
      integrators/home-assistant/meteo/manifest.json
  18. 506
      internal/database.go
  19. 681
      internal/database__test.go
  20. 12
      meteo.toml
  21. 16
      meteod.service
  22. 9
      meteod.toml
  23. 6
      utils/.gitignore
  24. 241
      utils/migrate_mysql.py
  25. 82
      web/api.html
  26. 47
      web/asset/Chart.css
  27. 14680
      web/asset/Chart.js
  28. 22
      web/asset/meteo.css
  29. 29
      web/dashboard.tmpl
  30. 66
      web/graph_t_hum.tmpl
  31. 26
      web/lightnings.tmpl
  32. 24
      web/ombrometer.tmpl
  33. 26
      web/ombrometer_create.tmpl
  34. 26
      web/ombrometer_edit.tmpl
  35. 24
      web/ombrometers.tmpl
  36. 21
      web/station.tmpl
  37. 27
      web/station_edit.tmpl

2
.gitignore vendored

@ -44,8 +44,6 @@
# Build files
build
build/
/meteo
/meteod
/meteo-influx-gateway
## Custom config files

35
Makefile

@ -1,41 +1,12 @@
default: all
all: meteo meteod influxgateway $(SUBDIRS)
all: meteo-influx-gateway
install: all
install meteo /usr/local/bin/
install meteod /usr/local/bin/
## ==== Easy requirement install ============================================ ##
# requirements for meteod (server)
req:
go get "github.com/BurntSushi/toml"
go get "github.com/gorilla/mux"
go get "github.com/mattn/go-sqlite3"
go get "github.com/eclipse/paho.mqtt.golang"
# requirements for meteo (client)
req-meteo:
go get "github.com/BurntSushi/toml"
req-gateway:
go get "github.com/BurntSushi/toml"
go get "github.com/jacobsa/go-serial/serial"
## === Builds =============================================================== ##
meteo: cmd/meteo/meteo.go
go build $^
meteod: cmd/meteod/meteod.go
go build $^
gateway: cmd/gateway/gateway.go
go build $^
influxgateway: cmd/influxgateway/meteo-mqtt-influxdb.go cmd/influxgateway/influxdb.go cmd/influxgateway/mqtt.go
go build -o meteo-influx-gateway $^
meteo-influx-gateway: cmd/meteo-influx-gateway/meteo-mqtt-influxdb.go cmd/meteo-influx-gateway/influxdb.go cmd/meteo-influx-gateway/mqtt.go
go build -o $@ $^
## === Tests ================================================================ ##
test: internal/database.go
go test -v ./...
$(SUBDIRS):
$(MAKE) -C $@

66
README.md

@ -4,51 +4,7 @@
Lightweight environmental monitoring solution
This project aims to provide a centralized environmental and room monitoring system for different sensors.
The meteo-daemon (`meteod`) runs on a centralised server instance, that collect all sensor data from different sensor nodes.
Client can attach to this server instance in order to read out the different readings of the sensors.
# Server
Requires `go >= 1.9.x` and the following repositories
go get "github.com/BurntSushi/toml"
go get "github.com/gorilla/mux"
go get "github.com/mattn/go-sqlite3"
go get "github.com/eclipse/paho.mqtt.golang"
A quick way of installing the requirements is
$ make req # Installs requirements
## Configuration
Currently manually. See `meteod.toml` for information
## Storage
`meteod` stores all data in a Sqlite3 database. By default `meteod.db` is taken, but the filename can be configured in `meteod.toml`.
# Client
There is currently a very simple CLI client available: `meteo`
meteo REMOTE
$ meteo http://meteo-service.local/
* 1 meteo-cluster 2019-05-14-17:24:01 22.51C|23.00 %rel| 95337hPa
## Build
Requires `go >= 1.9.x` and `"github.com/BurntSushi/toml"`
$ make req-meteo
$ make meteo
## Configuration
Currently manually. See `meteod.toml`
This project aims to provide a centralized environmental and room monitoring system for different sensors using mqtt. Data storage is supported via a `meteo-influxdb` gateway.
# Nodes/Sensors
@ -73,14 +29,22 @@ Every node that publishes `MQTT` packets in the given format is accepted.
PAYLOAD: {"node":1,"timestamp":0,"distance":12.1}
# if the timestamp is 0, the server replaces it with it's current time
## Data-Push via HTTP
# meteo-influx-gateway
The provided `meteo-influx-gateway` is a program to collect meteo data points from mqtt and push them to a influxdb database. This gateway is written in go and needs to be build
To push data via HTTP, you need a access `token`. The token identifies where the data belongs to.
go build ./...
The data is expected to be in the following `json` format
The gateway is configured via a [simple INI file](meteo-influx-gateway.ini.example):
{ "token":"test", "T":32.0, "Hum": 33.1, "P":1013.5 }
```ini
[mqtt]
remote = "127.0.0.1"
Example-`curl` script to push data to station 5
[influxdb]
remote = "http://127.0.0.1:8086"
username = "meteo"
password = "meteo"
database = "meteo"
```
$ curl 'http://localhost:8802/station/5' -X POST -H "Content-Type: application/json" --data { "token":"test", "T":32.0, "Hum": 33.1, "P":1013.5 }

2
cmd/gateway/.gitignore vendored

@ -1,2 +0,0 @@
# Binary
gateway

5
cmd/gateway/Makefile

@ -1,5 +0,0 @@
default: all
all: gateway
gateway: gateway.go
go build gateway.go

5
cmd/gateway/README.md

@ -1,5 +0,0 @@
# meteo Gateway
The meteo-gateway is a program to read current measurements from a serial port and push them to a `meteo` webserver
The use case is to have a Rasperry-Pi with `Arduino Nano` Sensors connected via serial port.

184
cmd/gateway/gateway.go

@ -1,184 +0,0 @@
/*
* Meteo program to read out connected ServerNode and push the data via HTTP to
* a given server
*/
package main
import (
"errors"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"strconv"
"strings"
"sync"
"github.com/BurntSushi/toml"
"github.com/jacobsa/go-serial/serial"
)
type Config struct {
Stations map[string]SerialStation `toml:"Serial"`
}
type SerialStation struct {
Device string
Baud uint
Remote string
Token string
}
type DataPoint struct {
Station int
Name string
Temperature float64
Humidity float64
Pressure float64
}
var cf Config
func parse(line string) (DataPoint, error) {
dp := DataPoint{}
line = strings.TrimSpace(line)
if line == "" {
return dp, nil
}
// XXX: Improve this: Espaced name
split := strings.Split(line, " ")
// Expected format: '1 "Meteo-Station" 2601 27 94366' (without '')
if len(split) != 5 {
return dp, errors.New("Illegal packet")
}
var err error
dp.Station, err = strconv.Atoi(split[0])
if err != nil {
return dp, err
}
dp.Name = split[1]
dp.Temperature, err = strconv.ParseFloat(split[2], 10)
if err != nil {
return dp, err
}
dp.Temperature /= 100.0
dp.Humidity, err = strconv.ParseFloat(split[3], 10)
if err != nil {
return dp, err
}
dp.Pressure, err = strconv.ParseFloat(split[4], 10)
if err != nil {
return dp, err
}
return dp, nil
}
func serialOpen(device string, baud uint) (io.ReadWriteCloser, error) {
// Set up options.
options := serial.OpenOptions{
PortName: device,
BaudRate: baud,
DataBits: 8,
StopBits: 1,
MinimumReadSize: 4,
}
// Open the port.
return serial.Open(options)
}
func handleSerial(station SerialStation) {
fmt.Println(station.Remote)
serial, err := serialOpen(station.Device, station.Baud)
if err != nil {
fmt.Fprintf(os.Stderr, "Error opening %s: %s\n", station.Device, err)
return
}
defer serial.Close()
data := make([]byte, 1024)
for { // Read continously
n, err := serial.Read(data)
if n == 0 {
break
}
if err != nil {
if err == io.EOF {
break
}
panic(err)
}
lines := strings.Split(string(data[:n]), "\n")
for _, line := range lines {
line = strings.TrimSpace(line)
dp, err := parse(line)
if err != nil {
fmt.Fprintf(os.Stderr, "Error reading: %s\n", err)
} else {
if dp.Station == 0 {
continue
} // Ignore, because invalid
err := publish(dp, station.Remote, station.Token)
if err != nil {
fmt.Fprintf(os.Stderr, "Error publishing: %s\n", err)
} else {
fmt.Println(line)
}
}
}
}
}
func main() {
// Configuration - set defaults and read from file
toml.DecodeFile("/etc/meteo/gateway.toml", &cf)
toml.DecodeFile("meteo.toml", &cf)
toml.DecodeFile("meteo-gateway.toml", &cf)
if len(cf.Stations) == 0 {
fmt.Fprintf(os.Stderr, "No station defined\n")
os.Exit(1)
}
// Create thread for each serial port
var wg sync.WaitGroup
for _, serial := range cf.Stations {
wg.Add(1)
go func(station SerialStation) {
defer wg.Done()
handleSerial(station)
}(serial)
}
wg.Wait()
}
func publish(dp DataPoint, remote string, token string) error {
// Build json packet
json := fmt.Sprintf("{\"token\":\"%s\",\"T\":%.2f,\"Hum\":%.2f,\"P\":%.2f}", token, dp.Temperature, dp.Humidity, dp.Pressure)
//fmt.Println(json)
// Prepare POST request
hc := http.Client{}
addr := fmt.Sprintf("%s/station/%d", remote, dp.Station)
req, err := http.NewRequest("POST", addr, strings.NewReader(json))
if err != nil {
return err
}
req.Header.Add("Content-Type", "application/json")
resp, err := hc.Do(req)
if err != nil {
return err
}
body, err := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
fmt.Fprintf(os.Stderr, "POST returned status %d\n%s\n\n", resp.StatusCode, string(body))
}
// All good
return nil
}

0
cmd/influxgateway/influxdb.go → cmd/meteo-influx-gateway/influxdb.go

0
cmd/influxgateway/meteo-mqtt-influxdb.go → cmd/meteo-influx-gateway/meteo-mqtt-influxdb.go

0
cmd/influxgateway/mqtt.go → cmd/meteo-influx-gateway/mqtt.go

297
cmd/meteo/meteo.go

@ -1,297 +0,0 @@
/*
* Simple CLI meteo client
*/
package main
import (
"errors"
"fmt"
"io/ioutil"
"net/http"
"os"
"strconv"
"strings"
"time"
"github.com/BurntSushi/toml"
)
// Terminal color codes
const KNRM = "\x1B[0m"
const KRED = "\x1B[31m"
const KGRN = "\x1B[32m"
const KYEL = "\x1B[33m"
const KBLU = "\x1B[34m"
const KMAG = "\x1B[35m"
const KCYN = "\x1B[36m"
const KWHT = "\x1B[37m"
type LocalStation struct {
Id int
Name string
Timestamp int64
Temperature float32
Humidity float32
Pressure float32
}
type StationMeta struct {
Id int
Name string
Location string
Description string
}
/* Warning and error ranges */
type Ranges struct {
T_Range []float32 `toml:"temp_range"`
Hum_Range []float32 `toml:"hum_range"`
//P_Range []float32 `toml:"p_range"`
}
type tomlClientConfig struct {
DefaultRemote string `toml:"DefaultRemote"`
Remotes map[string]Remote `toml:"Remotes"`
StationRanges map[string]Ranges `toml:"Ranges"`
}
type Remote struct {
Remote string
}
func httpGet(url string) ([]byte, error) {
ret := make([]byte, 0)
if len(url) == 0 {
return ret, errors.New("Empty URL")
}
resp, err := http.Get(url)
if err != nil {
return ret, err
}
if !strings.HasPrefix(resp.Status, "200 ") {
return ret, errors.New(fmt.Sprintf("Http error %s", resp.Status))
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
return body, err
}
func GetStations(baseUrl string) (map[int]StationMeta, error) {
ret := make(map[int]StationMeta, 0)
url := baseUrl + "/stations"
body, err := httpGet(url)
if err != nil {
return ret, err
}
response := strings.TrimSpace(string(body))
for _, line := range strings.Split(response, "\n") {
if len(line) == 0 || line[0] == '#' {
continue
}
params := strings.Split(line, ",")
if len(params) == 4 {
var err error
station := StationMeta{}
station.Id, err = strconv.Atoi(params[0])
if err != nil {
continue
}
station.Name = params[1]
station.Location = params[2]
station.Description = params[3]
ret[station.Id] = station
}
}
return ret, nil
}
func request(hostname string) ([]LocalStation, error) {
ret := make([]LocalStation, 0)
url := hostname
// Add https if nothing is there
autoHttps := false
if !strings.Contains(hostname, "://") {
autoHttps = true
url = "https://" + hostname + "/meteo"
}
if len(url) == 0 {
return ret, errors.New("Empty URL")
}
if url[len(url)-1] == '/' {
url = url[:len(url)-1]
}
body, err := httpGet(url + "/current")
if err != nil {
// Try with http instead of https, if we have added https automatically
if autoHttps {
//fmt.Fprintln(os.Stderr, " # Warning: Fallback to unencrypted http!")
url = strings.Replace(url, "https://", "http://", 1)
body, err = httpGet(url + "/current")
if err != nil {
return ret, err
}
} else {
return ret, err
}
}
stations, _ := GetStations(url) // Ignore errors here
response := strings.TrimSpace(string(body))
// Parse response lines
for _, line := range strings.Split(response, "\n") {
line = strings.TrimSpace(line)
if len(line) == 0 {
continue
}
if line[0] == '#' {
continue
}
params := strings.Split(line, ",")
if len(params) == 5 {
station := LocalStation{}
var err error
var f float64
station.Id, err = strconv.Atoi(params[0])
if err != nil {
continue
}
station.Timestamp, err = strconv.ParseInt(params[1], 10, 64)
if err != nil {
continue
}
f, err = strconv.ParseFloat(params[2], 32)
if err != nil {
continue
}
station.Temperature = float32(f)
f, err = strconv.ParseFloat(params[3], 32)
if err != nil {
continue
}
station.Humidity = float32(f)
f, err = strconv.ParseFloat(params[4], 32)
if err != nil {
continue
}
station.Pressure = float32(f)
station.Name = stations[station.Id].Name
ret = append(ret, station)
}
}
return ret, nil
}
func printHelp() {
fmt.Printf("Usage: %s [OPTIONS] REMOTE [REMOTE] ...\n", os.Args[0])
fmt.Println(" REMOTE defines a running meteo webserver")
fmt.Printf(" e.g. %s http://meteo.local/\n", os.Args[0])
fmt.Printf(" %s https://monitor-server.local/meteo/\n", os.Args[0])
fmt.Printf("\n")
fmt.Printf("OPTIONS\n")
fmt.Printf(" -h, --help Print this help message\n")
fmt.Printf("\n")
fmt.Println("https://github.com/grisu48/meteo")
}
func main() {
args := os.Args[1:]
hosts := make([]string, 0)
// Read configuration
var cf tomlClientConfig
toml.DecodeFile("/etc/meteo.toml", &cf)
toml.DecodeFile("meteo.toml", &cf)
// Parse arguments
for _, arg := range args {
if arg == "" {
continue
}
if arg[0] == '-' {
// Check for special parameters
if arg == "-h" || arg == "--help" {
printHelp()
os.Exit(0)
} else {
fmt.Fprintf(os.Stderr, "Illegal argument: %s\n", arg)
os.Exit(1)
}
} else {
hosts = append(hosts, arg)
}
}
if len(hosts) == 0 {
if cf.DefaultRemote == "" {
printHelp()
os.Exit(1)
} else {
hosts = append(hosts, cf.DefaultRemote)
}
}
for _, hostname := range hosts {
if val, ok := cf.Remotes[hostname]; ok {
hostname = val.Remote
}
fmt.Println(hostname)
stations, err := request(hostname)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %s\n", err)
}
for _, station := range stations {
timestamp := time.Unix(station.Timestamp, 0)
/* ==== Building the line with colors ==== */
// Yes, this has become a bit messy
fmt.Printf(" * %3d %-22s ", station.Id, station.Name)
// Set color for time
if time.Since(timestamp).Minutes() > 5 {
fmt.Printf(KRED)
} else {
fmt.Printf(KGRN)
}
fmt.Printf("%19s", timestamp.Format("2006-01-02-15:04:05"))
fmt.Printf(KNRM)
fmt.Printf(" ")
// Check if we have some custom ranges
if val, ok := cf.StationRanges[station.Name]; ok {
if len(val.T_Range) == 4 {
if station.Temperature <= val.T_Range[0] || station.Temperature >= val.T_Range[3] {
fmt.Printf(KRED)
} else if station.Temperature <= val.T_Range[1] || station.Temperature >= val.T_Range[2] {
fmt.Printf(KYEL)
} else {
fmt.Printf(KGRN)
}
}
fmt.Printf("%5.2fC", station.Temperature)
fmt.Printf(KNRM)
fmt.Printf("|")
if len(val.Hum_Range) == 4 {
if station.Humidity <= val.Hum_Range[0] || station.Humidity >= val.Hum_Range[3] {
fmt.Printf(KRED)
} else if station.Humidity <= val.Hum_Range[1] || station.Humidity >= val.Hum_Range[2] {
fmt.Printf(KYEL)
} else {
fmt.Printf(KGRN)
}
}
fmt.Printf("%5.2f %%rel", station.Humidity)
fmt.Printf(KNRM)
fmt.Printf("|%6.0fhPa\n", station.Pressure)
} else {
fmt.Printf("%5.2fC|%5.2f %%rel|%6.0fhPa\n", station.Temperature, station.Humidity, station.Pressure)
}
}
}
}

1482
cmd/meteod/meteod.go

File diff suppressed because it is too large Load Diff

6
go.mod

@ -1,14 +1,10 @@
module github.com/grisu48/meteo
go 1.11
go 1.14
require (
github.com/BurntSushi/toml v0.3.1
github.com/eclipse/paho.mqtt.golang v1.2.0
github.com/gorilla/mux v1.7.4
github.com/influxdata/influxdb1-client v0.0.0-20200827194710-b269163b24ab
github.com/jacobsa/go-serial v0.0.0-20180131005756-15cf729a72d4
github.com/mattn/go-sqlite3 v2.0.3+incompatible
github.com/smartystreets/goconvey v1.6.4 // indirect
golang.org/x/net v0.0.0-20200202094626-16171245cfb2 // indirect
gopkg.in/ini.v1 v1.62.0

8
go.sum

@ -1,19 +1,11 @@
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/eclipse/paho.mqtt.golang v1.2.0 h1:1F8mhG9+aO5/xpdtFkW4SxOJB67ukuDC3t2y2qayIX0=
github.com/eclipse/paho.mqtt.golang v1.2.0/go.mod h1:H9keYFcgq3Qr5OUJm/JZI/i6U7joQ8SYLhZwfeOo6Ts=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/mux v1.7.4 h1:VuZ8uybHlWmqV03+zRzdwKL4tUnIp1MAQtp1mIFE1bc=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/influxdata/influxdb1-client v0.0.0-20200827194710-b269163b24ab h1:HqW4xhhynfjrtEiiSGcQUd6vrK23iMam1FO8rI7mwig=
github.com/influxdata/influxdb1-client v0.0.0-20200827194710-b269163b24ab/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo=
github.com/jacobsa/go-serial v0.0.0-20180131005756-15cf729a72d4 h1:G2ztCwXov8mRvP0ZfjE6nAlaCX2XbykaeHdbT6KwDz0=
github.com/jacobsa/go-serial v0.0.0-20180131005756-15cf729a72d4/go.mod h1:2RvX5ZjVtsznNZPEt4xwJXNJrM3VTZoQf7V6gk0ysvs=
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/mattn/go-sqlite3 v2.0.3+incompatible h1:gXHsfypPkaMZrKbD5209QV9jbUTJKjyR5WD3HYQSd+U=
github.com/mattn/go-sqlite3 v2.0.3+incompatible/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s=

0
home-assistant/README.md → integrators/home-assistant/README.md

0
home-assistant/meteo/__init__.py → integrators/home-assistant/meteo/__init__.py

0
home-assistant/meteo/manifest.json → integrators/home-assistant/meteo/manifest.json

506
internal/database.go

@ -1,506 +0,0 @@
package meteo
import (
"database/sql"
"strconv"
_ "github.com/mattn/go-sqlite3"
)
/* Basic datapoint */
type DataPoint struct {
Timestamp int64
Station int
Temperature float32
Humidity float32
Pressure float32
}
/* Station */
type Station struct {
Id int
Name string
Location string
Description string
}
/* Ombrometer station reading */
type Rain struct {
Station int
Timestamp int64
Millimeters float32
}
type Persistence struct {
con *sql.DB
}
type Token struct {
Token string
Station int
}
type Lightning struct {
Station int
Timestamp int64
Distance float32
}
func OpenDb(filename string) (Persistence, error) {
db := Persistence{}
con, err := sql.Open("sqlite3", filename)
if err != nil {
return db, err
}
db.con = con
return db, nil
}
func (db *Persistence) Close() {
db.con.Close()
}
/** Get the highest id from the given table */
func (db *Persistence) getHighestId(table string, id string) (int, error) {
rows, err := db.con.Query("SELECT `" + id + "` FROM `" + table + "` ORDER BY `" + id + "` DESC LIMIT 1")
if err != nil {
return 0, err
}
defer rows.Close()
if rows.Next() {
var uid int
rows.Scan(&uid)
return uid, nil
}
return 0, nil
}
/* Prepare database, i.e. create tables ecc.
*/
func (db *Persistence) Prepare() error {
_, err := db.con.Exec("CREATE TABLE IF NOT EXISTS `stations` (`id` INT PRIMARY KEY, `name` VARCHAR(64), `location` TEXT, `description` TEXT);")
if err != nil {
return err
}
_, err = db.con.Exec("CREATE TABLE IF NOT EXISTS `tokens` (`token` VARCHAR(32) PRIMARY KEY, `station` INT);")
if err != nil {
return err
}
_, err = db.con.Exec("CREATE TABLE IF NOT EXISTS `lightnings` (`station` INT, `timestamp` INT, `distance` REAL, PRIMARY KEY(`station`,`timestamp`));")
if err != nil {
return err
}
_, err = db.con.Exec("CREATE TABLE IF NOT EXISTS `ombrometers` (`id` INT PRIMARY KEY, `name` VARCHAR(64), `location` TEXT, `description` TEXT);")
if err != nil {
return err
}
return nil
}
/** Get the station, the token is assigned to. Returns Token.Station = 0 if not token is found */
func (db *Persistence) GetToken(token string) (Token, error) {
ret := Token{Station: 0}
stmt, err := db.con.Prepare("SELECT `token`,`station` FROM `tokens` WHERE `token` = ? LIMIT 1")
if err != nil {
return ret, err
}
defer stmt.Close()
rows, err := stmt.Query(token)
if err != nil {
return ret, err
}
defer rows.Close()
if rows.Next() {
rows.Scan(&ret.Token, &ret.Station)
return ret, nil
}
return ret, nil
}
func (db *Persistence) GetStationTokens(station int) ([]Token, error) {
ret := make([]Token, 0)
stmt, err := db.con.Prepare("SELECT `token`,`station` FROM `tokens` WHERE `station` = ? LIMIT 1")
if err != nil {
return ret, err
}
defer stmt.Close()
rows, err := stmt.Query(station)
if err != nil {
return ret, err
}
defer rows.Close()
for rows.Next() {
tok := Token{}
rows.Scan(&tok.Token, &tok.Station)
ret = append(ret, tok)
}
return ret, nil
}
func (db *Persistence) InsertToken(token Token) error {
stmt, err := db.con.Prepare("INSERT OR IGNORE INTO `tokens` (`token`,`station`) VALUES (?,?);")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(token.Token, token.Station)
return err
}
func (db *Persistence) RemoveToken(token Token) error {
stmt, err := db.con.Prepare("DELETE FROM `tokens` WHERE `token` = ?")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(token.Token)
return err
}
/** Inserts the given station and assign the ID to the given station parameter */
func (db *Persistence) InsertStation(station *Station) error {
id := station.Id
var err error
if id == 0 {
id, err = db.getHighestId("stations", "id")
if err != nil {
return err
}
id += 1
}
// Create table
tablename := "station_" + strconv.Itoa(id)
_, err = db.con.Exec("CREATE TABLE IF NOT EXISTS `" + tablename + "` (`timestamp` INT PRIMARY KEY, `temperature` FLOAT, `humidity` FLOAT, `pressure` FLOAT);")
if err != nil {
return err
}
stmt, err := db.con.Prepare("INSERT INTO `stations` (`id`, `name`, `location`, `description`) VALUES (?,?,?,?);")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(id, station.Name, station.Location, station.Description)
if err != nil {
return err
}
station.Id = id
return nil
}
func (db *Persistence) GetStations() ([]Station, error) {
stations := make([]Station, 0)
rows, err := db.con.Query("SELECT `id`,`name`,`location`,`description` FROM `stations`")
if err != nil {
return stations, err
}
defer rows.Close()
for rows.Next() {
station := Station{}
rows.Scan(&station.Id, &station.Name, &station.Location, &station.Description)
stations = append(stations, station)
}
return stations, nil
}
func (db *Persistence) ExistsStation(id int) (bool, error) {
stmt, err := db.con.Prepare("SELECT `id` FROM `stations` WHERE `id` = ? LIMIT 1")
if err != nil {
return false, err
}
defer stmt.Close()
rows, err := stmt.Query(id)
if err != nil {
return false, err
}
defer rows.Close()
return rows.Next(), nil
}
func (db *Persistence) GetStation(id int) (Station, error) {
station := Station{}
stmt, err := db.con.Prepare("SELECT `id`,`name`,`location`,`description` FROM `stations` WHERE `id` = ? LIMIT 1")
if err != nil {
return station, err
}
defer stmt.Close()
rows, err := stmt.Query(id)
if err != nil {
return station, err
}
defer rows.Close()
if rows.Next() {
station := Station{}
rows.Scan(&station.Id, &station.Name, &station.Location, &station.Description)
return station, nil
} else {
station.Id = 0
return station, nil
}
}
func (db *Persistence) UpdateStation(station Station) error {
stmt, err := db.con.Prepare("UPDATE `stations` SET `name`=?,`location`=?,`description`=? WHERE `id` = ?")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(station.Name, station.Location, station.Description, station.Id)
return err
}
func (db *Persistence) GetLastDataPoints(station int, limit int) ([]DataPoint, error) {
datapoints := make([]DataPoint, 0)
tablename := "station_" + strconv.Itoa(station)
stmt, err := db.con.Prepare("SELECT `timestamp`,`temperature`,`humidity`,`pressure` FROM `" + tablename + "` ORDER BY `timestamp` DESC LIMIT ?")
if err != nil {
return datapoints, err
}
defer stmt.Close()
rows, err := stmt.Query(limit)
if err != nil {
return datapoints, err
}
defer rows.Close()
for rows.Next() {
datapoint := DataPoint{}
rows.Scan(&datapoint.Timestamp, &datapoint.Temperature, &datapoint.Humidity, &datapoint.Pressure)
datapoint.Station = station
datapoints = append(datapoints, datapoint)
}
return datapoints, nil
}
/** Query given station within the given timespan. If limit <=0, the limit is set to 100000 */
func (db *Persistence) QueryStation(station int, t_min int64, t_max int64, limit int64, offset int64) ([]DataPoint, error) {
datapoints := make([]DataPoint, 0)
tablename := "station_" + strconv.Itoa(station)
sql := "SELECT `timestamp`,`temperature`,`humidity`,`pressure` FROM `" + tablename + "` WHERE `timestamp` >= ? AND `timestamp` <= ? ORDER BY `timestamp` ASC LIMIT ? OFFSET ?"
stmt, err := db.con.Prepare(sql)
if err != nil {
return datapoints, err
}
defer stmt.Close()
if limit <= 0 {
limit = 100000
}
rows, err := stmt.Query(t_min, t_max, limit, offset)
if err != nil {
return datapoints, err
}
defer rows.Close()
for rows.Next() {
datapoint := DataPoint{}
rows.Scan(&datapoint.Timestamp, &datapoint.Temperature, &datapoint.Humidity, &datapoint.Pressure)
datapoint.Station = station
datapoints = append(datapoints, datapoint)
}
return datapoints, nil
}
/** Inserts the given datapoint to the database */
func (db *Persistence) InsertDataPoint(dp DataPoint) error {
tablename := "station_" + strconv.Itoa(dp.Station)
stmt, err := db.con.Prepare("INSERT OR REPLACE INTO `" + tablename + "` (`timestamp`,`temperature`,`humidity`,`pressure`) VALUES (?,?,?,?);")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(dp.Timestamp, dp.Temperature, dp.Humidity, dp.Pressure)
return err
}
/** Inserts the given lightning to the database */
func (db *Persistence) InsertLightning(light Lightning) error {
stmt, err := db.con.Prepare("INSERT OR IGNORE INTO `lightnings` (`station`,`timestamp`,`distance`) VALUES (?,?,?);")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(light.Station, light.Timestamp, light.Distance)
return err
}
func (db *Persistence) GetLightnings(limit int, offset int) ([]Lightning, error) {
ret := make([]Lightning, 0)
stmt, err := db.con.Prepare("SELECT `station`,`timestamp`,`distance` FROM `lightnings` ORDER BY `timestamp` DESC LIMIT ? OFFSET ?")
if err != nil {
return ret, err
}
defer stmt.Close()
rows, err := stmt.Query(limit, offset)
if err != nil {
return ret, err
}
defer rows.Close()
for rows.Next() {
lightning := Lightning{}
rows.Scan(&lightning.Station, &lightning.Timestamp, &lightning.Distance)
ret = append(ret, lightning)
}
return ret, err
}
func (db *Persistence) GetOmbrometers() ([]Station, error) {
stations := make([]Station, 0)
rows, err := db.con.Query("SELECT `id`,`name`,`location`,`description` FROM `ombrometers`")
if err != nil {
return stations, err
}
defer rows.Close()
for rows.Next() {
station := Station{}
rows.Scan(&station.Id, &station.Name, &station.Location, &station.Description)
stations = append(stations, station)
}
return stations, nil
}
/** Inserts the given ombrometer station and assign the ID to the given station parameter */
func (db *Persistence) InsertOmbrometer(station *Station) error {
id := station.Id
var err error
if id == 0 {
id, err = db.getHighestId("ombrometers", "id")
if err != nil {
return err
}
id++
}
// Create table
tablename := "ombrometer_" + strconv.Itoa(id)
_, err = db.con.Exec("CREATE TABLE IF NOT EXISTS `" + tablename + "` (`timestamp` INT PRIMARY KEY, `millimeter` FLOAT);")
if err != nil {
return err
}
stmt, err := db.con.Prepare("INSERT INTO `ombrometers` (`id`, `name`, `location`, `description`) VALUES (?,?,?,?);")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(id, station.Name, station.Location, station.Description)
if err != nil {
return err
}
station.Id = id
return nil
}
func (db *Persistence) UpdateOmbrometer(station Station) error {
stmt, err := db.con.Prepare("UPDATE `ombrometers` SET `name`=?,`location`=?,`description`=? WHERE `id` = ?")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(station.Name, station.Location, station.Description, station.Id)
return err
}
func (db *Persistence) GetOmbrometer(id int) (Station, error) {
station := Station{}
stmt, err := db.con.Prepare("SELECT `id`,`name`,`location`,`description` FROM `ombrometers` WHERE `id` = ? LIMIT 1")
if err != nil {
return station, err
}
defer stmt.Close()
rows, err := stmt.Query(id)
if err != nil {
return station, err
}
defer rows.Close()
if rows.Next() {
station := Station{}
rows.Scan(&station.Id, &station.Name, &station.Location, &station.Description)
return station, nil
} else {
station.Id = 0
return station, nil
}
}
// GetOmbrometerReadings returns the most recent readings from the given station
func (db *Persistence) GetOmbrometerReadings(station int, limit int) ([]Rain, error) {
datapoints := make([]Rain, 0)
tablename := "ombrometer_" + strconv.Itoa(station)
stmt, err := db.con.Prepare("SELECT `timestamp`,`millimeter` FROM `" + tablename + "` ORDER BY `timestamp` DESC LIMIT ?")
if err != nil {
return datapoints, err
}
defer stmt.Close()
rows, err := stmt.Query(limit)
if err != nil {
return datapoints, err
}
defer rows.Close()
for rows.Next() {
datapoint := Rain{}
rows.Scan(&datapoint.Timestamp, &datapoint.Millimeters)
datapoint.Station = station
datapoints = append(datapoints, datapoint)
}
return datapoints, nil
}
// GetOmbrometerLastReading returns the last reading of the given station or an empty new Rain object
func (db *Persistence) GetOmbrometerLastReading(station int) (Rain, error) {
dps, err := db.GetOmbrometerReadings(station, 1)
if err != nil || len(dps) == 0 {
return Rain{Station: station}, err
} else {
return dps[0], nil
}
}
// QueryOmbrometer queries the given station within the given timespan. If limit <=0, the limit is set to 100000
func (db *Persistence) QueryOmbrometer(station int, t_min int64, t_max int64, limit int64, offset int64) ([]Rain, error) {
datapoints := make([]Rain, 0)
tablename := "ombrometer_" + strconv.Itoa(station)
sql := "SELECT `timestamp`,`millimeter` FROM `" + tablename + "` WHERE `timestamp` >= ? AND `timestamp` <= ? ORDER BY `timestamp` ASC LIMIT ? OFFSET ?"
stmt, err := db.con.Prepare(sql)
if err != nil {
return datapoints, err
}
defer stmt.Close()
if limit <= 0 {
limit = 100000
}
rows, err := stmt.Query(t_min, t_max, limit, offset)
if err != nil {
return datapoints, err
}
defer rows.Close()
for rows.Next() {
datapoint := Rain{}
rows.Scan(&datapoint.Timestamp, &datapoint.Millimeters)
datapoint.Station = station
datapoints = append(datapoints, datapoint)
}
return datapoints, nil
}
// InsertRainMeasurement inserts a single rain measurement into the database
func (db *Persistence) InsertRainMeasurement(dp Rain) error {
tablename := "ombrometer_" + strconv.Itoa(dp.Station)
stmt, err := db.con.Prepare("INSERT OR REPLACE INTO `" + tablename + "` (`timestamp`,`millimeter`) VALUES (?,?);")
if err != nil {
return err
}
defer stmt.Close()
_, err = stmt.Exec(dp.Timestamp, dp.Millimeters)
return err
}

681
internal/database__test.go

@ -1,681 +0,0 @@
package meteo
import (
"fmt"
"log"
"math/rand"
"os"
"testing"
"time"
)
var testDatabase = "_test_meteod_.db"
var db Persistence
func randomInt(min int, max int) int {
if min > max {
return randomInt(max, min)
}
if min == max {
return min
}
return min + rand.Int()%(max-min)
}
func randomFloat32(min float32, max float32) float32 {
if min > max {
return randomFloat32(max, min)
}
if min == max {
return min
}
return min + rand.Float32()*(max-min)
}
func randomString(n int) string {
var letterRunes = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
b := make([]rune, n)
for i := range b {
b[i] = letterRunes[rand.Intn(len(letterRunes))]
}
return string(b)
}
func fileExists(filename string) bool {
if _, err := os.Stat(filename); err == nil {
return true
} else if os.IsNotExist(err) {
return false
} else {
return false // Assume it's a zombie or something
}
}
func TestMain(m *testing.M) {
// Initialisation
fmt.Println("Initializing test run ... ")
seed := time.Now().UnixNano()
rand.Seed(seed)
fmt.Printf("\tRandom seed %d\n", seed)
if fileExists(testDatabase) {
fmt.Fprintln(os.Stderr, "Test file already exists: "+testDatabase)
os.Exit(1)
}
fmt.Printf("\tFilename: %s\n", testDatabase)
var err error
db, err = OpenDb(testDatabase)
if err != nil {
fmt.Fprintln(os.Stderr, "Error setting up database: ", err)
os.Exit(1)
}
// Run tests
ret := m.Run()
// Cleanup
db.Close()
os.Remove(testDatabase)
os.Exit(ret)
}
func checkStation(station Station) bool {
ref := station
sec, err := db.GetStation(station.Id)
if err != nil {
fmt.Fprintf(os.Stderr, "Database error: %s\n", err)
return false
}
return ref == sec
}
func TestStations(t *testing.T) {
t.Log("Setting up database")
err := db.Prepare()
if err != nil {
t.Fatal("Error preparing database")
}
t.Log("Ensure database is empty")
stations, err := db.GetStations()
if err != nil {
t.Fatal("Database error: ", err)
}
if len(stations) > 0 {
t.Error(fmt.Sprintf("Fetched %d stations from expected empty database", len(stations)))
}
// Insert stations
t.Log("Inserting stations")
station1 := Station{Id: 0, Name: "Station 1", Description: "First test station"}
station2 := Station{Id: 0, Name: "Station 2", Description: "Second test station"}
err = db.InsertStation(&station1)
if err != nil {
t.Fatal("Database error: ", err)
}
stations, err = db.GetStations()
if err != nil {
t.Fatal("Database error: ", err)
}
if len(stations) != 1 {
t.Error(fmt.Sprintf("Fetched %d stations but expected 1", len(stations)))
}
if !checkStation(station1) {
t.Error("Comparison after inserting station 1 failed")
}
err = db.InsertStation(&station2)
if err != nil {
t.Fatal("Database error: ", err)
}
stations, err = db.GetStations()
if err != nil {
t.Fatal("Database error: ", err)
}
if len(stations) != 2 {
t.Error(fmt.Sprintf("Fetched %d stations but expected 2", len(stations)))
}
if !checkStation(station1) {
t.Error("Comparison after inserting station 2 failed")
}
t.Log("Checking for existing stations")
exists, err := db.ExistsStation(station1.Id)
if err != nil {
t.Fatal("Database error: ", err)
}
if !exists {
t.Error("Station 1 reported as not existing")
}
exists, err = db.ExistsStation(station2.Id)
if err != nil {
t.Fatal("Database error: ", err)
}
if !exists {
t.Error("Station 2 reported as not existing")
}
exists, err = db.ExistsStation(station1.Id + station2.Id)
if err != nil {
t.Fatal("Database error: ", err)
}
if exists {
t.Error("Station 1+2 reported as existing")
}
t.Log("Inserting station with certain ID")
station3 := Station{Id: 55, Name: "Station 55"}
err = db.InsertStation(&station3)
if err != nil {
t.Fatal("Database error: ", err)
}
if station3.Id != 55 {
t.Fatal("Inserting station_55 with id 55, but ID has been reset")
}
if !checkStation(station3) {
t.Error("Comparison after inserting station 55 failed")
}
t.Log("Updating station information")
station3.Name = "Station 55_1"
station3.Location = "Nowhere"
station3.Description = "Updated description"
err = db.UpdateStation(station3)
if err != nil {
t.Fatal("Database error: ", err)
}
station3_clone, err := db.GetStation(station3.Id)
if err != nil {
t.Fatal("Database error: ", err)
}