Patriot Geek: Web Server

Showing posts with label Web Server. Show all posts

Saturday, September 23, 2017

Exposing Data in a Captive Portal

So far, BME-280 sensor data has been exposed as a JSON API and as a web page. There are two problems with the second approach:

There must be a wifi router to which the ESP8266 can connect
The IP address the router assigns must be somehow displayed before we visit the page

In an earlier blog post, the IP address was displayed in the Arduino IDE's serial monitor. Of course, we could connect a display and output the IP address there. Wouldn't it be nice if our browser were automatically redirected to that page, sort of like what happens when you connect to a public network in your favorite coffee shop?

That's exactly what we'll do.

We establish our own access point such that whenever the user connects to that network, his browser will be automatically directed to a specific landing page. This landing page is called a "captive portal", and is typically used for user authentication, gather information, plant tracking cookies, etc.

Captive portals are not just for coffee shops any more - ours will display temperature, humidity, and air pressure from the BME-280 sensor!

Here's the code:

/**
 * BME280-Captive-Portal
 * 
 * By: Mike Klepper
 * Date: 23 September 2017
 * 
 * This program exposes BME-280 data as a captive portal.
 * 
 * See patriot-geek.blogspot.com
 * for explanation.
 */

#include "ESP8266WiFi.h"
#include "DNSServer.h"
#include "ESP8266WebServer.h"
#include "Adafruit_Sensor.h"
#include "Adafruit_BME280.h"

const int STARTUP_DELAY = 500;

const char* MESSAGE_503 = "503 Service Unavailable";

const char* AP_NAME = "BME-280 Data";
const byte DNS_PORT = 53;
const IPAddress SUBNET_MASK(255, 255, 255, 0);
const IPAddress AP_IP(192, 168, 1, 1);
const byte WEB_SERVER_PORT = 80;

boolean sensorAvailable;

DNSServer dnsServer;
ESP8266WebServer webServer(WEB_SERVER_PORT);

Adafruit_BME280 bme;

void setup(void)
{
  // Start the BME280 sensor
  if(!bme.begin())
  {
    sensorAvailable = false;
  }
  else
  {
    sensorAvailable = true;
    delay(STARTUP_DELAY);
  }

  WiFi.mode(WIFI_AP);
  WiFi.softAPConfig(AP_IP, AP_IP, SUBNET_MASK);
  WiFi.softAP(AP_NAME);

  dnsServer.start(DNS_PORT, "*", AP_IP);

  webServer.onNotFound([]() 
  {
    returnHtml();
  });
  
  webServer.begin();
}

void loop(void)
{
  dnsServer.processNextRequest();
  webServer.handleClient();
  yield();
}

void returnHtml()
{
  if(sensorAvailable)
  {
    // Get values
    float tempC = bme.readTemperature();
    float humidity = bme.readHumidity();
    float pressurePascals = bme.readPressure();
  
    // Convert to British units
    float tempF = 9.0/5.0 * tempC + 32.0;
    float pressureInchesOfMercury = 0.000295299830714 * pressurePascals;
  
    // Build HTML response
    String responseHtml = "";
    responseHtml += "<!DOCTYPE html>";
    responseHtml += "<html>";
    responseHtml += "    <head>";
    responseHtml += "        <meta charset=\"UTF-8\">";
    responseHtml += "        <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">";
    responseHtml += "        <title>BME-280 Sensor Data</title>";
    responseHtml += "        <style>";
    responseHtml += "            body {font-family: sans-serif}";
    responseHtml += "            h1 {font-size: 1.0cm}";
    responseHtml += "            p {font-size: 0.50cm}";
    responseHtml += "            button {font-size: 1.0cm}";
    responseHtml += "        </style>";
    responseHtml += "        <script type=\"text/javascript\">";
    responseHtml += "        function refreshPage()";
    responseHtml += "        {";
    responseHtml += "            location.reload(true);";
    responseHtml += "        }";
    responseHtml += "        </script>";
    responseHtml += "    </head>";
    responseHtml += "    <body>";
    responseHtml += "        <h1>BME-280 Sensor Data</h1>";
    responseHtml += "        <p>Temperature: " + String(tempF) + "&deg; F</p>";
    responseHtml += "        <p>Humidity: " + String(humidity) + "%</p>";
    responseHtml += "        <p>Pressure: " + String(pressureInchesOfMercury) + " inHg</p>";
    responseHtml += "        <button type=\"button\" onclick=\"refreshPage()\">Refresh</button>";
    responseHtml += "    </body>";
    responseHtml += "</html>";
  
    // Return HTML with correct MIME type
    webServer.send(200, "text/html; charset=utf-8", responseHtml);
  }
  else
  {
    webServer.send(500, "text/plain", MESSAGE_503);
  }
}

Once it is running, you will see a new wireless network, called "BME-280 Data".

Connect to it, and you're immediately presented with the sensor data!

The only new part of the code are on lines 49 through 58:

49	Set the ESP8266 into access point mode
50	Set the AP's local IP address, gateway IP, and subnet mask
51	Specify the AP's name
53	Start a domain name server
55-58	Return the same HTML regardless of the address the user tries to visit

This technique - creating a captive portal - has many applications beyond displaying sensor data. In particular, it can be used to view and edit configuration values. This will be explored in a future post.

Why bother having the user connect to that AP - why not just make the AP name to be the sensor data? One problem with this is that the AP name will usually be truncated by the network manager of the computer or smart phone. Further, most computers or smart phones cache the names of the networks it finds.

Saturday, September 2, 2017

Exposing Data in a Web Page

In a previous post, BME-280 data was exposed as JSON, and that data was made available using a simple API. Once the web server was created and API URL paths were specified, exposing sensor data as JSON became just a problem in string manipulation - we just constructed a string that contained the data, and that string was in valid JSON format.

This same trick can be applied to exposing data in an HTML page - we'll construct a string that includes the sensor data, and that string will be a valid HTML5 document. We return that string with mime type text/html, and it will be rendered in a browser!

Why stop with HTML? We can also use string manipulation to include CSS and JavaScript!

Our goal is to have the ESP8266 serve a page that:

Presents temperature, humidity, and barometric pressure from the BME-280 sensor
Includes CSS that makes the page somewhat attractive on both notebook computers and iPhones
Has button that when clicked will call a JavaScript function that refreshes the page

We could add this to the program in that previous post, but to keep things simple, this functionality will be put in a separate program.

Here's the program:

/**
 * BME280-WebServer-HTML
 * 
 * By: Mike Klepper
 * Date: 2 September 2017
 * 
 * This program exposes BME-280 data in a web page.
 * 
 * See patriot-geek.blogspot.com
 * for explanation.
 */

#include "ESP8266WiFi.h"
#include "WiFiClient.h"
#include "ESP8266WebServer.h"
#include "Adafruit_Sensor.h"
#include "Adafruit_BME280.h"

const char* SSID = "**********";
const char* PASSWORD = "**********";

const int DELAY = 3000;
const int STARTUP_DELAY = 500;

const char* MESSAGE_404 = "404 Not Found";
const char* MESSAGE_503 = "503 Service Unavailable";

boolean sensorAvailable;

ESP8266WebServer server(80);
Adafruit_BME280 bme;

void setup(void)
{
  Serial.begin(115200);

  // Start the BME280 sensor
  if(!bme.begin())
  {
    Serial.println("Could not find a valid BME280 sensor, check wiring!");
    sensorAvailable = false;
  }
  else
  {
    sensorAvailable = true;
    delay(STARTUP_DELAY);
  }

  WiFi.begin(SSID, PASSWORD);

  // Wait for the connection
  while(WiFi.status() != WL_CONNECTED) 
  {
    delay(STARTUP_DELAY);
    Serial.print(".");
  }
  
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(SSID);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  server.on("/", returnHtml);
  server.on("", returnHtml);
  
  server.onNotFound(return404Error);

  server.begin();
  Serial.println("HTTP server started");
}

void loop(void)
{
  server.handleClient();
  yield();
}

void return404Error()
{
  server.send(404, "text/plain", MESSAGE_404);
}

void return503Error()
{
  server.send(500, "text/plain", MESSAGE_503);
}

void returnHtml()
{
  if(sensorAvailable)
  {
    // Get values
    float tempC = bme.readTemperature();
    float humidity = bme.readHumidity();
    float pressurePascals = bme.readPressure();
  
    // Convert to British units
    float tempF = 9.0/5.0 * tempC + 32.0;
    float pressureInchesOfMercury = 0.000295299830714 * pressurePascals;
  
    // Build HTML response
    String responseHtml = "";
    responseHtml += "<!DOCTYPE html>";
    responseHtml += "<html>";
    responseHtml += "    <head>";
    responseHtml += "        <meta charset=\"UTF-8\">";
    responseHtml += "        <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">";
    responseHtml += "        <title>BME-280 Sensor Data</title>";
    responseHtml += "        <style>";
    responseHtml += "            body {font-family: sans-serif}";
    responseHtml += "            h1 {font-size: 1.0cm}";
    responseHtml += "            p {font-size: 0.50cm}";
    responseHtml += "            button {font-size: 1.0cm}";
    responseHtml += "        </style>";
    responseHtml += "        <script type=\"text/javascript\">";
    responseHtml += "        function refreshPage()";
    responseHtml += "        {";
    responseHtml += "            location.reload(true);";
    responseHtml += "        }";
    responseHtml += "        </script>";
    responseHtml += "    </head>";
    responseHtml += "    <body>";
    responseHtml += "        <h1>BME-280 Sensor Data</h1>";
    responseHtml += "        <p>Temperature: " + String(tempF) + "&deg; F</p>";
    responseHtml += "        <p>Humidity: " + String(humidity) + "%</p>";
    responseHtml += "        <p>Pressure: " + String(pressureInchesOfMercury) + " inHg</p>";
    responseHtml += "        <button type=\"button\" onclick=\"refreshPage()\">Refresh</button>";
    responseHtml += "    </body>";
    responseHtml += "</html>";
  
    // Return HTML with correct MIME type
    server.send(200, "text/html; charset=utf-8", responseHtml);
  }
  else
  {
    return503Error();
  }
}

Once the program is running, the IP address that the network assigns to the ESP8266 is displayed in the serial monitor. For example, my IP address is (currently) 192.168.1.8. Open a browser and go to http://192.168.1.8, and the following will be displayed:

On the iPhone, the result is:

Click or tap the "Refresh" button and the page does indeed refresh.

This program functions very similar to the one in the previous post, so I won't do a code walk-through.

Most any web developer will complain about the way I mixed HTML, CSS, and JavaScript in one file. But that is the attitude of one who has the luxury of great bandwidth and fast servers, and we are not in that position.

Monday, April 17, 2017

Exposing Data as a REST API

How to expose the temperature, pressure, and humidity data over WiFi? There are two subquestions here:

What network mechanism to use?
What request and response formats to use?

For the network mechanism, we will create a small web server on the network that will listen for REST requests and return the data formatted as JSON. In other words, we will finally be using the ESP8266's WiFi capabilities! YAY!!

What should the request URIs for our REST API look like? Ask ten developers this question, and be prepared to get fifteen answers! The only thing they'll agree upon is that we will only be making GET requests, since we can only read data from a sensor.

We will structure the request URLs as follows:

/home/living-room/temperature /home/living-room/humidity /home/living-room/barometric-pressure /home/living-room/ /home/living-room

This last two requests will return all data that is available from the BME280 sensor; both are included so as to demonstrate using one handler for both URIs. We will also handle 404 and 503 errors.

We will return the data as JSON, but how exactly should that JSON should be formatted? Put ten developers in a room, ask them that question, and expect only two developers to leave that room alive!

There are two extremes: return the minimum amount of requested data, or return all that data plus a ton of metadata, resource links, etc. This latter approach will not only require us to add a significant number of characters to the JSON, but will also require additional URIs. This is called HATEOAS.

We will take the minimal approach here. Further, we will always return data in British units. For example, in response to a GET request to home/living-room/temperature, we will return:


{
    "temperature": 86.56
}

The Code
The following code will connect to the local network, and create a web server listening on port 80:

/**
 * BME280-WebServer-JSON
 * 
 * By: Mike Klepper
 * Date: 17 April 2017
 * 
 * This program exposes BME280 data as a REST API.
 * 
 * See blog post on patriot-geek.blogspot.com
 * for instructions.
 */

#include "ESP8266WiFi.h"
#include "WiFiClient.h"
#include "ESP8266WebServer.h"
#include "Adafruit_Sensor.h"
#include "Adafruit_BME280.h"

const char* SSID = "**********";
const char* PASSWORD = "**********";

const int DELAY = 3000;
const int STARTUP_DELAY = 500;

const char* MESSAGE_404 = "404 Not Found";
const char* MESSAGE_503 = "503 Service Unavailable";

boolean sensorAvailable;

ESP8266WebServer server(80);
Adafruit_BME280 bme;

void setup(void)
{
  Serial.begin(115200);

  // Start the BME280 sensor
  if(!bme.begin())
  {
    Serial.println("Could not find a valid BME280 sensor, check wiring!");
    sensorAvailable = false;
  }
  else
  {
    sensorAvailable = true;
    delay(STARTUP_DELAY);
  }

  WiFi.begin(SSID, PASSWORD);

  // Wait for the connection
  while(WiFi.status() != WL_CONNECTED) 
  {
    delay(STARTUP_DELAY);
    Serial.print(".");
  }
  
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(SSID);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  server.on("/home/living-room/temperature", returnTemperature);
  server.on("/home/living-room/humidity", returnHumidity);
  server.on("/home/living-room/barometric-pressure", returnPressure);
  server.on("/home/living-room/", returnAll);
  server.on("/home/living-room", returnAll);
  
  server.onNotFound(return404Error);

  server.begin();
  Serial.println("HTTP server started");
}

void loop(void)
{
  server.handleClient();
  yield();
}

void return404Error()
{
  server.send(404, "text/plain", MESSAGE_404);
}

void return503Error()
{
  server.send(500, "text/plain", MESSAGE_503);
}

void returnTemperature() 
{
  if(sensorAvailable)
  {
    // Get temperature and format it in degree Fahrenheit
    float tempC = bme.readTemperature();
    float tempF = 9.0/5.0 * tempC + 32.0;
  
    // Build JSON response
    String responseJson = "{\n\"temperature\":" + String(tempF) + "}";
  
    // Return JSON with correct MIME type
    server.send(200, "application/json", responseJson);
  }
  else
  {
    return503Error();
  }
}

void returnHumidity() 
{
  if(sensorAvailable)
  {
    // Get humidity
    float humidity = bme.readHumidity();
  
    // Build JSON response
    String responseJson = "{\n\"humidity\":" + String(humidity) + "}";
  
    // Return JSON with correct MIME type
    server.send(200, "application/json", responseJson);
  }
  else
  {
    return503Error();
  }
}

void returnPressure() 
{
  if(sensorAvailable)
  {
    // Get pressure and change units to inHg
    float pressurePascals = bme.readPressure();
    float pressureInchesOfMercury = 0.000295299830714 * pressurePascals;
  
    // Build JSON response
    String responseJson = "{\n\"barometric-pressure\":" + String(pressureInchesOfMercury) + "}";
  
    // Return JSON with correct MIME type
    server.send(200, "application/json", responseJson);
  }
  else
  {
    return503Error();
  }
}

void returnAll()
{
  if(sensorAvailable)
  {
    // Get values
    float tempC = bme.readTemperature();
    float humidity = bme.readHumidity();
    float pressurePascals = bme.readPressure();
  
    // Convert to British units
    float tempF = 9.0/5.0 * tempC + 32.0;
    float pressureInchesOfMercury = 0.000295299830714 * pressurePascals;
  
    // Build JSON response
    String responseJson = "";
    responseJson += "{";
    responseJson +=     "\"temperature\":" + String(tempF) + ",";
    responseJson +=     "\"humidity\":" + String(humidity) + ",";
    responseJson +=     "\"barometric-pressure\":" + String(pressureInchesOfMercury);
    responseJson += "}";
  
    // Return JSON with correct MIME type
    server.send(200, "application/json", responseJson);
  }
  else
  {
    return503Error();
  }
}

Running the Code
When this code is ran, the IP address that the network assigns to the ESP8266 is reported in the serial monitor. For example, my IP address is (currently) 10.200.206.75. To test this code, then, start a browser or other HTTP client and go to the URL made of that IP address together with one of the paths. For example when I visit http://10.200.206.75/home/living-room, the result is:

What happened to the quotes? Some browser extensions, like JSONView for Chrome, suppresses the quotes and adds tabs. If this extension is disabled, the result looks like this:

Code Explanation

13 - 15	Include libraries for working with WiFi as well as creating a web server on an ESP8266
16 - 17	Include the libraries for using the BME280 sensor
19 - 20	Specify the network name (SSID) and password for the network we're connecting to. Everybody's password is "**********", no?
25 - 26	Messages for 404 and 503 errors
38 - 47	Initialize the BME280 sensor; set `sensorAvailable` to the appropriate value
49	Connect to the network
52 - 56	Print dots to the serial monitor until we're connected
58 - 62	Print connection info to the serial monitor
64 - 66	Specify the paths used to request single sensor readings (temperature only, for example), and associate a handler for each
67 - 68	Associate the same handler - `returnAll()` - for two different paths
70	Have web server call `return404Error()` whenever a URI handler is not specified
72	Start the web server!
76 - 80	Handle incoming requests, repeatedly
82 - 85	Whenever a path hasn't been found, send HTTP error code 404, using MIME type "text/plain"
87 - 90	For internal errors (like when the BME280 isn't available), send HTTP error code 503, using MIME type "text/plain"
92 - 110	Callback for handling `/home/living-room/temperature` requests
94	If the sensor is available at startup...
96 - 98	Read the temperature from the BME280 and convert to Fahrenheit
101	Wrap the temperature inside a JSON object
104	Return that JSON using MIME type "application/json" and response code 200
106 - 109	If the sensor is NOT available, return 503.
112 - 129	Callback for handling `/home/living-room/humidity` requests
131 - 149	Callback for handling `/home/living-room/barometric-pressure` requests
151 - 179	Callback for handling `/home/living-room/` and `/home/living-room` requests

Notes:

The problem with setting sensorAvailable at the start is: what happens if the BME280 becomes disconnected later? This can be handled by performing the check for bme.begin() in each of the response handlers.
To connect to a wifi network that doesn't use a password, change line 50 to read:
WiFi.begin(SSID);
Change lines 26 and 27 to give more interesting error messages!