Monday, December 31, 2018

ESP8266 library for EFM8 microcontrollers - Server mode

In previous article ESP8266 library for EFM8 microcontrollers was presented the ESP8266 module and how can be interfaced with a microcontroller in client mode. This article covers ESP8266 in server mode and how to send commands to ESP8266 over the internet.

Of course this is not a real web server although with an SD card whole HTML pages could be served. The purpose of this library is to extract queries from a URL requested by a client. This way for example you could change color and brightness of an RGB led using a color wheel in a phone app or control home appliances from anywhere in the world.




ESP8266 library functions in server mode

ESP_StartServer(const char *port)

Function
Start the ESP8266 server at the specified port
Arguments
port: string of characters that denotes the port number used by web clients to connect to the ESP. The port must be forwarded in router. I believe by default ESP8266 uses the 333 port but port 80 is better
Returns
true if successful or false if not
 
ESP_DeleteServer(void)

Function
Deletes the server
Arguments
void
Returns
true if successful or false if not

ESP_ClientConnected(void)

Function
Check if a client is connected to the server and return true or false
Arguments
none
Returns
true if successful or false if not

ESP_GetQueryData(query_names[], query_values_return[NR_OF_URL_QUERIES][URL_QUERY_VALUE_LENGTH])

Function
Takes a list of URL query names and extract their values in the provided array
Arguments
query_names: (char *) a pointer to an array with query names. See the example bellow
query_values_return: (char) a 2D array in which the query values corresponding to provided query names will be storred
Returns
true if URL queries were found or false if not. Web browsers will ask for a favicon and if this returns false a 404 response can be sent

ESP_ServerReply(html_data[], HTTP_Headers)

Function
Send an HTTP response to the client together with some HTML content
Arguments
html_data: (char) HTML content. It can also be a null character
HTTP_Headers: (char *) the HTTP response. The following macros are available HTTP_200_OK or HTTP_404_NOT_FOUND
Returns
true if successful or false if not




In the main c file first add the following defines. It is important to read the description of each define and set them properly.

main.c
//-----------------------------------------------------------------------------
// Defines
//-----------------------------------------------------------------------------
#define F_CPU        24500000 // CPU FREQUENCY IN Hz
#define BAUD_RATE    115200

/* Server mode */
// How many queries to search in URL. ex: http://192.168.100.5:333?var1=10&var2=4
// In this example there are 2 queries names - var1 and var2
#define NR_OF_URL_QUERIES   3 // rows

// How many characters will query value have (after =)
// For example if var2 will have only 1 character but var2 2 characters, then the greatest length is taken
// In this case 3 plus 1 for null character so we have 4
#define URL_QUERY_VALUE_LENGTH   4 // columns (add 1 for null terminator)

// How many characters has the longest query name. var1 has 4 characters
#define URL_QUERY_NAME_LENGTH   1

Now some includes

#include <SI_EFM8BB1_Register_Enums.h> // SFR declarations
#include "globals.h"
#include "delay.h"
#include "ESP8266.h"
#include "uart.h"

Links for these files can be found in the main article here ESP8266 library for EFM8 microcontrollers.


Example on how to use the ESP8266 library in server mode, setup the EFM8BB1 microcontroller

This example describes how to control the brightness and color of an RGB led over the internet by extracting the query values and printing them on the terminal. The Termite terminal can be used for testing purposes.

main.c

//-----------------------------------------------------------------------------
// Defines
//-----------------------------------------------------------------------------
#define F_CPU        24500000 // CPU FREQUENCY IN Hz
#define BAUD_RATE    115200

/* Server mode */
// How many queries to search in URL. For example to change the color and brightness of an RGB led: 
// http://192.168.100.5:301?r=10&g=4&b=49&a=100
// In this example there are 4 query names - r, g, b and a
#define NR_OF_URL_QUERIES   4 // rows

// How many characters will query value have (after =)
// For example if r will have only 1 character but g 2 characters, then the greatest length is taken
// In this case 3 plus 1 for null character so we have 4
#define URL_QUERY_VALUE_LENGTH   4 // columns (add 1 for null terminator)

// How many characters has the longest query name. r, g, b and a are all 1 character long
#define URL_QUERY_NAME_LENGTH   1

//-----------------------------------------------------------------------------
// Includes
//-----------------------------------------------------------------------------
#include <SI_EFM8BB1_Register_Enums.h> // SFR declarations
#include "globals.h"
#include "delay.h"
#include "ESP8266.h"
#include "uart.h"

//-----------------------------------------------------------------------------
// Global VARIABLES
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
// Global CONSTANTS
//-----------------------------------------------------------------------------
SI_SBIT(LED, SFR_P0, 0); // LED on Port 0: Bit 0

//-----------------------------------------------------------------------------
// Function PROTOTYPES
//-----------------------------------------------------------------------------
void Port_Init(void);

//-----------------------------------------------------------------------------
// SiLabs_Startup() Routine
// ----------------------------------------------------------------------------
// This function is called immediately after reset, before the initialization
// code is run in SILABS_STARTUP.A51 (which runs before main() ). This is a
// useful place to disable the watchdog timer, which is enable by default
// and may trigger before main() in some instances.
//-----------------------------------------------------------------------------
void SiLabs_Startup(void){
 // Disable Watchdog with key sequence
 WDTCN = 0xDE; // First key
 WDTCN = 0xAD; // Second key
}


//-----------------------------------------------------------------------------
// main() Routine
// ----------------------------------------------------------------------------
// Note: the software watchdog timer is not disabled by default in this
// example, so a long-running program will reset periodically unless
// the timer is disabled or your program periodically writes to it.
//-----------------------------------------------------------------------------
int main(void){
 //-------------------------
 // Local variables
 // ------------------------
 // Server mode
 // Data from URL queries will be stored here
 char query_values[NR_OF_URL_QUERIES][URL_QUERY_VALUE_LENGTH];
 // Query names to search for in the incoming URL
 char *URL_queries[] = {"r", "g", "b", "a"};

 bool esp_setup_ok = false;

 //-------------------------
 // Initialization
 // ------------------------
 Port_Init();
 UART_Begin(BAUD_RATE);

 // Delay added for debugging purposes. Can be removed.
 _delay_ms(2000);

 // Set ESP in server mode
 esp_setup_ok = ESP_Setup(STATION_MODE, SERVER);


 //-------------------------
 // while loop
 // ------------------------
 while(1){
  // Check if there is an incoming connection
  if(ESP_ClientConnected()){
   // Extract the values from incoming URL using provided queries (URL_queries) and store the data in the provided array (query_values)
   if(ESP_GetQueryData(URL_queries, query_values)){
    
    // Send a HTTP 200 OK response and some HTML data
    ESP_ServerReply("<h1>ESP OK</h1>", HTTP_200_OK);

    // Print on the terminal received RGB values for debugging purposes
    // These values could be used to control an RGB LED but first the values must be converted from
    // string to integer using stringToInt() function located in globals.h
    UART_Send("\nRED: ");
    UART_Send(query_values[0]);
    UART_Send("\nGREEN: ");
    UART_Send(query_values[1]);
    UART_Send("\nBLUE: ");
    UART_Send(query_values[2]);
    UART_Send("\n");
    UART_Send("\nBRIGHTNESS: ");
    UART_Send(query_values[3]);
    UART_Send("\n");
   }else{
    // The function will return false when no queries were found in URL
    // Browsers will send a request for favicon so here you could send an embedded message as a favicon or return 404
    // In any case a response must be sent
    ESP_ServerReply("\0", HTTP_404_NOT_FOUND);
   }
   }
  }
}


//-----------------------------------------------------------------------------
// Functions
//-----------------------------------------------------------------------------
void Port_Init(void){
 // -------------------------------------------------------------
 // 1. Select the input mode (analog or digital) for all port pins
 // using the Port Input Mode register (PnMDIN)
 // -------------------------------------------------------------

 // --- Configure a pin as a digital input
 // -------------------------------------------------------------
 // 1.1 Set the bit associated with the pin in the PnMDIN register to 1.
 // This selects digital mode for the pin.
 //P0MDIN |= (1<<0);

 // 2.1 Clear the bit associated with the pin in the PnMDOUT register to 0.
 // This configures the pin as open-drain.
 //P0MDOUT &= ~(1<<0);

 // 3.1 Set the bit associated with the pin in the Pn register to 1.
 // This tells the output driver to “drive” logic high. Because the pin is
 // configured as open-drain, the high-side driver is disabled,
 // and the pin may be used as an input.
 //P0 |= (1<<0);


 // --- To configure a pin as a digital, push-pull output:
 // -------------------------------------------------------------
 // 1.1 Set the bit associated with the pin in the PnMDIN register to 1.
 // This selects digital mode for the pin.
 P0MDIN |= (1<<0);


 // 2.1 Set the bit associated with the pin in the PnMDOUT register to 1.
 // This configures the pin as push-pull.
 P0MDOUT |= (1<<0);


 // --- To configure a pin as analog, the following steps should be taken:
 // -------------------------------------------------------------
 // 1.1 Clear the bit associated with the pin in the PnMDIN register to 0.
 // This selects analog mode for the pin.
 //P0MDIN &= ~(1<<0);

 // 2.1 Set the bit associated with the pin in the Pn register to 1.
 //P0 |= (1<<0);

 // 3.1 Skip the bit associated with the pin in the PnSKIP register
 // to ensure the crossbar does not attempt to assign a function to the pin.
 //P0SKIP |= (1<<0);

 // -------------------------------------------------------------
 // 2. Select any pins to be skipped by the I/O crossbar
 // using the Port Skip registers (PnSKIP)
 // -------------------------------------------------------------
 //P0SKIP = 0xFF; // Skip all port pins
 //P1SKIP = 0xFF; // Skip all port pins


 // -------------------------------------------------------------
 // 3. Assign port pins to desired peripherals
 // -------------------------------------------------------------

 // -------------------------------------------------------------
 // 4. Enable crossbar and weak pull-ups
 // -------------------------------------------------------------
 XBR2 |= 0x40;

 // Make all pins low to save power. The pins to be used can be set afterwards
 //P0 = 0;
 //P1 = 0;
 //P2 = 0;

 // SYSCLK - CPU speed
 CLKSEL = CLKSEL_CLKSL__HFOSC | CLKSEL_CLKDIV__SYSCLK_DIV_1;
}


//-----------------------------------------------------------------------------
// Interrupts
//-----------------------------------------------------------------------------






Saturday, December 8, 2018

ESP8266 library for EFM8 microcontrollers

ESP8266 is a low-cost serial to Wi-Fi module that is very popular among electronics hobbyists. It can be used to connect a microcontroller to internet over Wi-Fi. The things you can do with it are numerous. You could monitor and log various sensor data like atmospheric pressure, temperature and humidity together with geo locations on websites like thingspeak.com. Or you could retrieve information from internet such as weather, number of subscribers on YouTube, send tweets with your microcontroller, send email notification if your plants get dry, etc.


The module consists mainly of a 32-bit microcontroller with full TCP/UDP stack support and an external flash memory chip from 0.5MB to 8MB depending on the module type. The cheapest one is ESP8266-01.
The module can be used in two ways:
1. With an external microcontroller that sends commands to ESP using so called AT commands. This is the simplest way. Communication between ESP and microcontroller is done via UART. A serial terminal such as Termite terminal can be used to manually test the module.
2. The ESP can also be programmed and used without an external MCU since already contains a powerful MCU. The easiest way to program it is by using Arduino IDE with appropriate libraries. Once the chip is reprogrammed it cannot take AT commands anymore. The disadvantage is that the cheap modules have only one or two general purpose pins so you can't do much without an external microcontroller.

ESP8266 pinout and power

Power

The voltage to the ESP8266 must not exceed 3.3V on supply and communication lines. So make sure that the USB to UART or the microcontroller used for interface also uses 3.3V or use a voltage level translator. Also the power supply must be able to source at least 170mA - the peak current of the ESP.

Some modules are not breadboard compatible. You can use an adapter or simply temporarily solder some wire jumpers to the pins and connect them to breadboard for testing or use DuPont connectors (recommended).

To communicate with the ESP8266 via a serial terminal you can make a USB to serial adapter like in this post.

Pinout

On the back side of the ESP there is the marking for the pins as follow:

Pin Number Pin Name Alternate Name Normally used for Alternate purpose
1 GND
Power ground
2 TX GPIO–1 UART Transmit.
Connected to Rx pin of programmer/u
Can act as a General purpose
input/output pin when not used as TX
3 IO2
General purpose input/output pin 2
4 EN
Chip Enable – active high
5 IO0 Flash General purpose input/output pin 0 Takes module into serial programming
when held low during start up
6 RST
Resets the module
7 RX GPIO-3 UART Receive
Connected to Tx pin of programmer/uC
Can act as a General purpose
input/output pin when not used as RX
8 3V3
VCC. Connect to +3.3V only

Schematic

ESP8266 schematic

R16 is a 10k resistor used to pull-up the Reset pin. VCC and EN (enable) pins are connected to 3.3V. The Reset pin is connected also to a momentary switch and when pressed it resets the ESP. GPIO2 can be connected to a header but is optional. Same for GPIO0 except this pin is pulled high by R1 a 10k resistor. SW32 is another momentary switch that pulls GPIO0 to ground when pressed and is used only for programming (flashing) the chip. If you are using only AT commands, the two switches can be omitted.

ESP8266 library using AT commands

This library depends on the following UART library that can be downloaded from here or from the end of this page.
For now this library is only made for EFM8BB1 Busy Bee and EFM8BB3 microcontrollers but in the future it will be adapted for AVR also.

In the ESP8266.h file there are a few optional settings

#define DEBUGGING               1 // default 0

If set to 1 some error messages will be displayed over UART in case of setup errors.
Debugging tip: when testing the interface between the MCU and ESP you can use the Rx (receive0 pin of an USB to serial adapter to see the communication messages between the two. If you put the Rx pin on Tx of the microcontroller then you can see what MCU is transmitting. If Rx is connected to Rx of the microcontroller then you can see what ESP8266 is transmitting.

#define USE_SERVER_MODE  1

Set this to 0 if not using the ESP in server mode, to save some space.

For setting the ESP8266 in server mode read this article ESP8266 library for EFM8 microcontrollers - Server mode

Library setup

In your main c file add the following defines. These defines must be added before the includes

main.c
//-----------------------------------------------------------------------------
// Defines
//-----------------------------------------------------------------------------
#define F_CPU                     24500000 // CPU FREQUENCY IN Hz
#define BAUD_RATE                 115200
#define RECEIVED_DATA_ARRAY_SIZE  41 // size of array where to store the incoming data

BAUD_RATE is the speed that the microcontroller will communicate with the ESP. Usually 9600 or 115200 is used. If you see gibberish in the serial terminal, try another baud rate.
RECEIVED_DATA_ARRAY_SIZE is the size of the array declared later bellow, in which the received data from internet will be stored. Be sure to leave 1 byte for the null character that indicates the end of string.

Then include the following header files. The links with the necessary files can be found at the end of the page.

main.c
//-----------------------------------------------------------------------------
// Includes
//-----------------------------------------------------------------------------
#include <SI_EFM8BB1_Register_Enums.h> // SFR declarations
#include "globals.h"
#include "delay.h"
#include "uart.h" #include "ESP8266.h"

Library Functions


ESP_Setup(tcp_ip_mode, ssid, key)

Function:
sets up the UART at declared baud rate, checks if ESP module is responding and then sets up the Wi-Fi mode, TCP/IP mode and connects the ESP to the router with offered credentials
Arguments:
tcp_ip_mode: one of the available macros CLIENT or SERVER
ssid:
key: (char *) SSID and KEY of the router. The key must be between 8 and 64 bytes ASCII. Can be left empty like this "" useful in some scenarios
Returns:
(bool) true if successful or false if not

ESP_Setup_AP(wifi_mode, tcp_ip_mode, ap_ssid, ap_key)

Function:
Sets up an access point with an SSID and key given by the user and with WPA_WPA2_PSK encryption on channel 1
Arguments:
wifi_mode: one of the available macros ACCESS_POINT_MODE

ACCESS_POINT_MODE - ESP is used to connect other devices to the Internet through router
tcp_ip_mode: one of the available macros CLIENT or SERVER
ap_ssid:
ap_key: (char *) desired SSID and KEY of the access point. The key must be between 8 and 64 bytes ASCII
Returns:
(bool) true if successful or false if not

ESP_Connect(linkID, type, remoteIP, remotePort, UDPLocalPort, UDPMode)

Function:
Connects to a domain or IP to send or retrieve data
Arguments:
linkID: (char *) ID of network connection (0~4), used for multiple connections
type: connection type. Available macros: TCP or UDP 
remoteIP: (char *) string that represents a domain or IP to connect to. Do not append http://
remotePort: (char *) the remote port number
UDPLocalPort: (char *) optional. UDP port of the ESP8266. Pass an empty string if not used, like this ""
UDPMode: (char *) In the UDP transparent transmission, the value of this parameter has to be 0.
       0: the destination peer entity of UDP will not change; this is the default setting.
       1: the destination peer entity of UDP can change once.
       2: the destination peer entity of UDP is allowed to change.

Returns:
true if successful or false if not

ESP_SendData(linkID, payload, dataLength)

Function:
Sends data after ESP_Connect() is executed. Must run immediately after that.
Arguments:
linkID: (char *) ID of network connection (0~4), used for multiple connections
payload: (char *) string or array that represents the payload. Can also be a GET request like this: "GET http://192.168.100.2 HTTP/1.0\r\n\r\n"
dataLength: (uint8_t) number of bytes/length of the payload
Returns:
true if successful or false if not

ESP_ParseAndCheck(keywords[],  match_all, uppercase_all, timeout_ms, data_ret[], array_size)

Function:
used by the above two functions to check the response from the ESP8266 if it contains certain keywords like CONNECT OK or ERROR.
After ESP_Connect() exits, the ESP will start dumping data in the UART Rx buffer so this function must be executed as soon as possible in order to process the incoming data.
This function is very useful to extract some data between two keywords. Also the user can provide up to 4 keywords and it will return true or false if one or all keywords are found in the incoming data from the internet
Arguments:
keywords: (char *) a pointer to an array of up to 4 keywords to be searched. See the examples bellow to see how to use this
match_all: (bool) 1 or 0. If 1, the function will return true only if all the keywords are found in the incoming string of data. If you want to know if only one keyword was found then use 0 
uppercase_all: (bool) if true then the keywords and the incoming data will be converted to uppercase characters. Is useful when say for example you want to find OK and the data contains Ok. Without uppercase the function will return false - not found 
timeout_ms: (uint16_t) timeout in milliseconds. Depending of how much data you expect to receive set this to a few seconds. If the keywords are found the function will exit before timeout. If the keywords are not found the function will wait until timeout 
data_ret: (unsigned char) an array that user provides in which the data between two keywords will be stored 
array_size: (uint8_t) the size of the data_ret array. The array must be sized to fit expected data + 1 for null character. If set to 0 the data will not be returned in the data_ret array. This way data_ret can be initiated with only 1 byte in size and the function used to just return true or false for keywords match
Returns:
true if a keyword or all keywords are found - depending of match_all argument, false if not

ESP_UpdateQueryString(URL[], query, new_value, nrOfDigits)

Function:
Modifies the values of  a URL query string. The values bust be of the type integer. Used for say if you have some readings like temperature, voltage values... and you want to send them in a URL quey. See examples bellow
Arguments:
URL: (char) the URL 
query: (char *) the URL query string including = for which the value needs to be modified 
value: (int16_t) the new value after the = sign 
nrOfDigits: (uint8_t) how many digits is the value formed of
Returns:
void

linkID is not fully implemented and should be always 0. The ESP8266 can support up to 5 connections anyway. Also CIPMUX must be 1 for multiple connections by using SERVER_MODE macro as ESP_Setup() argument.

Extra Functions


ESP_DisconnectAP(void)

Function:
Disconnects the ESP from the access point (router). To reconnect it again, run ESP_Setup() 
Arguments:
none
Returns:
(bool) true if successful or false if not
 
ESP_FactoryReset(void)

Function:
Reset the ESP device to factory settings
Arguments:
none
Returns:
(bool) true if successful or false if not
 

Practical Examples

Log temperature and humidity on thingspeak.com using ESP8266

In the following code are presented two examples of how to use this ESP8266 library.
URL_1 is used to log temperature and humidity on thingspeak.com. You can use the api keys that I provided and see the results by visiting this public channel that I've made for testing purposes https://thingspeak.com/channels/429357.

URL_2 is another example that uses an app from thingspeak to receive a command string. This string is extracted from the received data and the HTTP headers and other unnecessary data is discarded. This way you can parse megabytes of received data and keep only a small part that matters, in a small array.

A small delay of a few milliseconds should be added between connections because after the connection is established, the ESP has to receive at least some HTTP headers in response. After that you can initiate a new connection.
Also depending of what sites you use for IoT, there needs to be a waiting time between connections like 5 seconds to prevent the site being flooded with too many connections in a short time.

main.c
//-----------------------------------------------------------------------------
// Includes
//-----------------------------------------------------------------------------
#include <SI_EFM8BB1_Register_Enums.h> // SFR declarations
#include "globals.h"
#include "delay.h"
#include "ESP8266.h"
#include "uart.h"

//-----------------------------------------------------------------------------
// Defines
//-----------------------------------------------------------------------------
#define F_CPU          24500000 // CPU FREQUENCY IN Hz
#define BAUD_RATE        115200
#define RECEIVED_DATA_ARRAY_SIZE   41 // size of array where to store the incoming data

//-----------------------------------------------------------------------------
// Global VARIABLES
//-----------------------------------------------------------------------------
int8_t field1_temperature = 27;
int8_t field2_humidity = 65;

//-----------------------------------------------------------------------------
// Global CONSTANTS
//-----------------------------------------------------------------------------
// URLs (IMPORTANT: don't forget to add the HTTP headers after the URL HTTP/1.0\r\n\r\n)
char xdata URL_1[] = "GET /update?api_key=2CY50F6OFB6RWP06&field1=0000&field2=0000 HTTP/1.0\r\n\r\n";
char xdata URL_2[] = "GET /talkbacks/23510/commands/14569888.json?api_key=0JAXMR1PR130D6XM HTTP/1.0\r\n\r\n";
const char *URL_1_DOMAIN = "api.thingspeak.com";
const char *ESP_PORT = "80";

//-----------------------------------------------------------------------------
// Function PROTOTYPES
//-----------------------------------------------------------------------------
void Port_Init(void);

//-----------------------------------------------------------------------------
// SiLabs_Startup() Routine
// ----------------------------------------------------------------------------
// This function is called immediately after reset, before the initialization
// code is run in SILABS_STARTUP.A51 (which runs before main() ). This is a
// useful place to disable the watchdog timer, which is enable by default
// and may trigger before main() in some instances.
//-----------------------------------------------------------------------------
void SiLabs_Startup(void){
  // Disable Watchdog with key sequence
  WDTCN = 0xDE; // First key
  WDTCN = 0xAD; // Second key
}


//-----------------------------------------------------------------------------
// main() Routine
// ----------------------------------------------------------------------------
// Note: the software watchdog timer is not disabled by default in this
// example, so a long-running program will reset periodically unless
// the timer is disabled or your program periodically writes to it.
//-----------------------------------------------------------------------------
int main(void){
  //-------------------------
  // Local variables
  // ------------------------
  uint8_t xdata receivedData[RECEIVED_DATA_ARRAY_SIZE] = {'\0'};
  // Number of keywords in this case is 2 but the array must be of size 3 so one byte is for null terminator
  char *keywords[3] = {"command_string\":\"", "\",\"position"};
  bool esp_setup_ok = false;
  bool esp_url_sent_ok = false;
 
  //-------------------------
  // Initialization
  // ------------------------
  Port_Init();
 
  _delay_ms(2000);
 
  esp_setup_ok = ESP_Setup(STATION_MODE, CLIENT);
 
  // Update and send measured temperature and humidity
  if(esp_setup_ok){
  // Update the URL's query string with a new value 
  ESP_UpdateQueryString(URL_1, "field1=", field1_temperature, URL_VALUE_NR_OF_DIGITS);
  
  // Update the URL's query string with a new value
  ESP_UpdateQueryString(URL_1, "field2=", field2_humidity, URL_VALUE_NR_OF_DIGITS);
  
  // Send measured temperature and humidity
  ESP_Connect(URL_1_DOMAIN, ESP_PORT, URL_1);
  
  _delay_ms(1000);
  
  // Receive and parse data from an URL
  if(ESP_Connect(URL_1_DOMAIN, ESP_PORT, URL_2)){
   // Incoming data will be
   // {"id":14569888,"command_string":"Important string that needs to be parsed","position":1,"executed_at":null,"created_at":"2018-11-29T03:46:03Z"}
   // and we want to extract this: Important string that needs to be parsed
   // So the first keyword will be <command_string":"> and the second keyword will be <","position> The " must be escaped by putting \ in front of them
   if(ESP_ParseAndCheck(keywords, true, false, 10000, receivedData, RECEIVED_DATA_ARRAY_SIZE)){
    // Display received and parsed data
    UART_Send("\n");
    UART_Send(receivedData);
    UART_Send("\nKeyword found\n");
   }else{
    UART_Send("\nKeyword not found\n");
   }
  }
 }
 
 
 // Some servers don't allow many queries in a short time so add the necessary delay
 _delay_ms(2000);
 
 // Change the keywords if necessary
 keywords[0] = "id\":";
 keywords[1] = ",\"";
 
 // Receive and parse data from another URL
 if(esp_setup_ok){
  if(ESP_Connect(URL_1_DOMAIN, ESP_PORT, URL_2)){
   // Incoming data will be
   // {"id":14569888,"command_string":"Important string that needs to be parsed","position":1,"executed_at":null,"created_at":"2018-11-29T03:46:03Z"}
   // and we want to extract id: 14569888
   // So the first keyword will be <id":> and the second keyword will be <,"> The " must be escaped by putting \ in front of them
   if(ESP_ParseAndCheck(keywords, false, false, 10000, receivedData, RECEIVED_DATA_ARRAY_SIZE)){ 
    // Display received and parsed data
    UART_Send("\n");
    UART_Send(receivedData);
    UART_Send("\nKeyword found\n");
   }else{
    UART_Send("\nKeyword not found\n");
   }
  }
 }
 
 // Update and send measured temperature and humidity
 if(esp_setup_ok){
  field1_temperature = -5;
  field2_humidity = 55;
  
  // Update the URL's query string with a new value
  ESP_UpdateQueryString(URL_1, "field1=", field1_temperature, URL_VALUE_NR_OF_DIGITS);
  
  // Update the URL's query string with a new value
  ESP_UpdateQueryString(URL_1, "field2=", field2_humidity, URL_VALUE_NR_OF_DIGITS);
  
  // Send measured temperature and humidity
  esp_url_sent_ok = ESP_Connect(URL_1_DOMAIN, ESP_PORT, URL_1);
 }
 
  //-------------------------
  // while loop
  // ------------------------
  while(1){
 
 }
}


//-----------------------------------------------------------------------------
// Functions
//-----------------------------------------------------------------------------
void Port_Init(void){
 // -------------------------------------------------------------
 // 1. Select the input mode (analog or digital) for all port pins
 // using the Port Input Mode register (PnMDIN)
 // -------------------------------------------------------------
 
 // --- Configure a pin as a digital input
 // -------------------------------------------------------------
 // 1.1 Set the bit associated with the pin in the PnMDIN register to 1.
 // This selects digital mode for the pin.
 //P0MDIN |= (1<<0);
 
 // 2.1 Clear the bit associated with the pin in the PnMDOUT register to 0.
 // This configures the pin as open-drain.
 //P0MDOUT &= ~(1<<0);
 
 // 3.1 Set the bit associated with the pin in the Pn register to 1.
 // This tells the output driver to “drive” logic high. Because the pin is
 // configured as open-drain, the high-side driver is disabled,
 // and the pin may be used as an input.
 //P0 |= (1<<0);
 
 
 // --- To configure a pin as a digital, push-pull output:
 // -------------------------------------------------------------
 // 1.1 Set the bit associated with the pin in the PnMDIN register to 1.
 // This selects digital mode for the pin.
 P0MDIN |= (1<<0);
 
 
 // 2.1 Set the bit associated with the pin in the PnMDOUT register to 1.
 // This configures the pin as push-pull.
 P0MDOUT |= (1<<0);
 
 
 // --- To configure a pin as analog, the following steps should be taken:
 // -------------------------------------------------------------
 // 1.1 Clear the bit associated with the pin in the PnMDIN register to 0.
 // This selects analog mode for the pin.
 //P0MDIN &= ~(1<<0);
 
 // 2.1 Set the bit associated with the pin in the Pn register to 1.
 //P0 |= (1<<0);
 
 // 3.1 Skip the bit associated with the pin in the PnSKIP register
 // to ensure the crossbar does not attempt to assign a function to the pin.
 //P0SKIP |= (1<<0);
 
 // -------------------------------------------------------------
 // 2. Select any pins to be skipped by the I/O crossbar
 // using the Port Skip registers (PnSKIP)
 // -------------------------------------------------------------
 
 // -------------------------------------------------------------
 // 3. Assign port pins to desired peripherals
 // -------------------------------------------------------------
 
 // -------------------------------------------------------------
 // 4. Enable crossbar and weak pull-ups
 // -------------------------------------------------------------
 XBR2 |= 0x40;
 
 // SYSCLK - CPU speed
 CLKSEL = CLKSEL_CLKSL__HFOSC | CLKSEL_CLKDIV__SYSCLK_DIV_1;
}


//-----------------------------------------------------------------------------
// Interrupts
//-----------------------------------------------------------------------------


Simple example

main.c

int main(void){

    bool esp_setup_ok = false;
    
    // Wait for ESP to connect to wifi
    _delay_ms(10000);
    
    // Set ESP in client mode
    esp_setup_ok = ESP_Setup(STATION_MODE, CLIENT, "YOUR_SSID", "YOUR_KEY");
    
    while(1){
    
    }

}

You can leave questions or report bugs in the comment section bellow.

Download

v2.1
+ support for server mode
+ UDP mode
- fixed some errors in the search functions


Other resources

delay.h

Setting the ESP8266 in server mode
ESP8266 library for EFM8BB1 Busy Bee microcontrollers - Server mode

AT instruction set
https://www.espressif.com/sites/default/files/documentation/4a-esp8266_at_instruction_set_en.pdf


Thursday, July 12, 2018

AVR EEPROM Library | ATmega328P

Sometimes some data needs to be saved after the microcontroller is powered off. Say you have a rotary encoder to set audio volume. When the user sets the volume, the value is stored in SRAM but when the power goes off, the memory content is lost. In this cases you would use the EEPROM memory to store data. AVR devices have three types of memory:
  • Flash memory - for the program code
  • SRAM memory - also referred as data memory (volatile memory)
  • EEPROM memory


EEPROM stands for Electronically Erasable Read-Only Memory and is a non-volatile memory, meaning that unlike RAM, it will hold the stored information even after power is removed, much like an USB flash drive. Here can be stored settings and data that can not be hard coded, usually taken by the interaction with a user. Keep in mind that the EEPROM has a lifespan of 100,000 writes - reads are unlimited - so is not a good idea to write to this memory every second or even every few seconds.

Normally, to make use of the EEPROM memory, you would have to read the datasheet and implement some read/write functions using some registers. But fear not - the AVR-GCC compiler comes with prebuilt routines making things much easier.

How to use the AVR EEPROM Library

Include the library

#include <avr/eeprom.h>

There are five main types of EEPROM access: byte, word, dword, float and block. Each type has three types of functions: write, update, and read.
In AVR-GCC, a word is two bytes long and a double word (dword) is 4 bytes, while a block is an arbitrary number of bytes which you supply.

uint8_t eeprom_read_byte ( const uint8_t * addr )
void eeprom_write_byte ( uint8_t *addr, uint8_t value )
void eeprom_update_byte ( uint8_t *addr, uint8_t value )

uint16_t eeprom_read_word ( const uint16_t * addr )
void eeprom_write_word ( uint16_t *addr, uint16_t value )
void eeprom_update_word ( uint16_t *addr, uint16_t value )

uint32_t eeprom_read_dword ( const uint32_t * addr )
void eeprom_write_dword ( uint32_t *addr, uint32_t value )
void eeprom_update_dword ( uint32_t *addr, uint32_t value )

float eeprom_read_float ( const float * addr )
void eeprom_write_float ( float *addr, float value )
void eeprom_update_float ( float *addr, float value )

void eeprom_read_block ( void * pointer_ram, const void * pointer_eeprom, size_t n)
void eeprom_write_block ( const void * pointer_ram, void * pointer_eeprom, size_t n)
void eeprom_update_block ( const void * pointer_ram, void * pointer_eeprom, size_t n)

It is highly recommended that update functions should be used instead of the write functions. Update functions will first check if the written data differs from the one already in EEPROM and only then it writes, and so increasing the EEPROM lifetime. It's a bit slower than the write function, because it executes read and then write, but is not like you write to EEPROM every few milliseconds, so it shouldn't matter.



Reading data from the EEPROM


Reading a byte


uint8_t byteFromEEPROM;
byteFromEEPROM = eeprom_read_byte((uint8_t*)10);

This will read out location 10 of the EEPROM, and put it into the variable byteFromEEPROM. The function expects an pointer to an address, and because 10 is a constant we typecast it. The address can be from 0 to maximum EEPROM size.

Reading a word


uint16_t wordFromEEPROM;
wordFromEEPROM = eeprom_read_word((uint16_t*)10);

Same as before, except the data type is of two bytes long now.
For dword and float functions you would use a pointer to a uint32_t or float variable.

Reading a block from EEPROM


uint8_t stringOfData[10];
eeprom_read_block((void*)&stringOfData, (const void*)12, 10)

The block command are useful when a larger number of bytes need to be read/write.

eeprom_read_block functions don't return anything; instead they modify the provided buffer stringOfData. In our case the data type for stringOfData array is uint8_t but the function expects a void pointer. A void pointer allows the function to have flexibility meaning it can work with any data types. But you must be sure that the type of data in EEPROM is the same as the provided array buffer; you can't put a uint16_t in a uint8_t variable.

(void*)&stringOfData
 
Typecast our array buffer to a void pointer. Notice the & operator. This means that is passing the address of the array not the data that it stores. In case of arrays the & operator it's optional because an array is always a pointer to it's first index (e.g stringOfData[0]), but in case of a regular variable, is not optional.

(const void*)12

Read the content starting at address 12. Because nothing is modified here the constant address number is typecast to a constant void pointer.

10

How many bytes to read. In our case, read 10 bytes starting at address 12 and put them in the stringOfData array buffer. Make sure the array is big enough to fit all the bytes. This parameter can also be a variable instead of a constant.

Writing data to the EEPROM


Writing a byte


uint8_t byteToEEPROM;
byteToEEPROM = 100;
eeprom_update_byte((uint8_t*)10, byteToEEPROM);

As in the case of read command, the first argument is the EEPROM address, typecast to a uint8_t pointer, except that now it takes a second argument - the data to be written to EEPROM.

Writing a word


uint16_t wordToEEPROM;
wordToEEPROM = 2600;
eeprom_update_word((uint16_t*)10, wordToEEPROM);

And double words and floats can be written using the eeprom_update_dword() and eeprom_update_float() functions and a uint32_t or float variable.
 

Writing a block to EEPROM


uint8_t stringOfData[10] = "Hello";
eeprom_update_block((const void*)&stringOfData, (void*)12, sizeof(stringOfData));

The first parameter is the data to be written to the EEPROM and it can be an array or even a struct, and is of the type const since the array is not being modified as in the case of read command.
Second parameter is the EEPROM address from where the string of data will start to be written.
Third and last argument is the length of the array in bytes obtained using sizeof function because if we decide to modify the size of stringOfData we wouldn't need to remember to change the size argument of the function.



Writing variables to EEPROM using the EEMEM atribute

Keeping track of all those addresses is hard and messy. Wouldn't be better to use variable names instead of addresses to refer to EEPROM locations? You can do this using EEMEM attribute placed before the variable name, and the compiler will take care of where in EEPROM memory should place them.

Bellow is an example on how to use the EEMEM attribute. First some variable are declared in the global scope with the EEMEM attribute and are prefixed with "eeprom_" to distinguish them from their counterpart variables located in RAM.
At startup the firmware loads the values from EEPROM into RAM.
In the while loop the code could verify if any variables have changed and then update the EEPROM with the new values.
This is the same as explained above, except instead of using addresses we are using the addresses of variables. Notice the & operator before the variable name; we are not passing the value of the variable but the address of it.

/*************************************************************
 INCLUDES
**************************************************************/
#include <avr/io.h>
#include <avr/eeprom.h>
#include <util/delay.h>
#include <string.h>


// variables stored on EEPROM using EEMEM attribute. Must be global.
uint8_t EEMEM eeprom_phoneVolume = 100;
uint16_t EEMEM eeprom_receivedMessages = 0;
float EEMEM eeprom_temperature = 0.0;
char EEMEM eeprom_lastSMS[40] = "Pizza order received";
   

/*************************************************************
 MAIN FUNCTION
**************************************************************/
int main(void){
   // variables stored on SRAM
   uint8_t phoneVolume = 75;
   uint16_t receivedMessages;
   float temperature;
   char lastSMS[40];
   
   // Update variable located in RAM, using last saved values inside EEPROM
   phoneVolume = eeprom_read_byte(&eeprom_phoneVolume);
   receivedMessages = eeprom_read_word(&eeprom_receivedMessages);
   temperature = eeprom_read_float(&eeprom_temperature);
   eeprom_read_block((void*)lastSMS, (const void*)eeprom_lastSMS, sizeof(lastSMS));
 
   while(1){
      phoneVolume = 45;
      receivedMessages = 1;
      temperature = 27.5;
      strcpy(lastSMS, "Pizza delivered");
      
      //_delay_ms(1000);
      
      // if volume changed
      eeprom_update_byte(&eeprom_phoneVolume, phoneVolume);
      
      // if number of received messages changed
      eeprom_update_word(&eeprom_receivedMessages, receivedMessages);
      
      // update temperature
      eeprom_update_float(&eeprom_temperature, temperature);

      // if new message is received
      eeprom_update_block((const void*)lastSMS, (void*)eeprom_lastSMS, sizeof(lastSMS));
   }
}

You could also update only a part of an array. In the following example, the array is updated starting from index 2 with the size argument as 3; so the array will be modified starting from index 2 to index 4 ((2 + 3) - 1).

uint8_t stringOfData[10] = "Hello";

eeprom_update_block((const void*)&stringOfData[2], (void*)&eeprom_stringOfData[2], 3);

Setting EEPROM variables with initial/default values

If, when declaring a variable with the EEMEM attribute you assign a value to it, the compiler will generate an .eep file. This file can be manually uploaded to the EEPROM or depending on the programmer, it will be automagically uploaded. Initially all EEPROM memory defaults to 0xFF.

Writing structures on EEPROM using EEMEM attribute

To keep the code more organized, struct data types can also be written to EEPROM as a whole or individual variables.

/*************************************************************
 INCLUDES
**************************************************************/
#include <avr/io.h>
#include <avr/eeprom.h>
#include <util/delay.h>
#include <string.h>

// structure stored in EEPROM. Must be declared in global space
typedef struct {
 uint8_t eeprom_phoneVolume;
 uint16_t eeprom_receivedMessages;
 float eeprom_temperature;
 char eeprom_lastSMS[40];
}eeprom_struct;

eeprom_struct EEMEM eeprom_phoneSettings;

/*************************************************************
 MAIN FUNCTION
**************************************************************/
int main(void){ 
   // structure stored in RAM
   struct {
    uint8_t phoneVolume;
    uint16_t receivedMessages;
    float temperature;
    char lastSMS[40];
   }phoneSettings;
   
   // Update variable located in RAM, using last saved values inside EEPROM
   phoneSettings.phoneVolume = eeprom_read_byte(&eeprom_phoneSettings.eeprom_phoneVolume);
   phoneSettings.receivedMessages = eeprom_read_word(&eeprom_phoneSettings.eeprom_receivedMessages);
   phoneSettings.temperature = eeprom_read_float(&eeprom_phoneSettings.eeprom_temperature);
   eeprom_read_block((void*)phoneSettings.lastSMS, (const void*)eeprom_phoneSettings.eeprom_lastSMS, sizeof(phoneSettings.lastSMS));
 
   while(1){
      phoneSettings.phoneVolume = 45;
      phoneSettings.receivedMessages = 1;
      phoneSettings. temperature = 27.5;
      strcpy(phoneSettings.lastSMS, "Pizza delivered");
   
 //_delay_ms(1000);
      
      // if volume changed
      eeprom_update_byte(&eeprom_phoneSettings.eeprom_phoneVolume, phoneSettings.phoneVolume);
      
      // if number of received messages changed
      eeprom_update_word(&eeprom_phoneSettings.eeprom_receivedMessages, phoneSettings.receivedMessages);
      
      // update temperature
      eeprom_update_float(&eeprom_phoneSettings.eeprom_temperature, phoneSettings.temperature);

      // if new message is received
      eeprom_update_block((const void*)&phoneSettings.lastSMS, (void*)&eeprom_phoneSettings.eeprom_lastSMS, sizeof(phoneSettings.lastSMS));
   
 // or... put whole structure on EEPROM
 eeprom_update_block((const void*)&phoneSettings, (void*)&eeprom_phoneSettings, sizeof(phoneSettings));
   }
}



Other things to consider

  • When using a bootloader to program the MCU via USB, you must first check if SELFPRGEN in SPMCSR register is zero before attempting to write to EEPROM. For more info about this check the datasheet.
  •  The EEPROM functions should not be used in an ISR function and outside at the same time because if a function in the main is interrupted by an interrupt, and another function inside the ISR attempts to access the EEPROM, both are working with the same EEPROM registers and will result in a mess of corrupted data. In this scenario, the interrupts should be disabled while accessing the EEPROM and then re-enabled.
  • All of the read/write functions first make sure the EEPROM is ready to be accessed using the eeprom__busy_wait() loop function. So for time critical applications, the state of the eeprom should be polled using eeprom_is_ready() function and if the return value is 1 then the eeprom read/write functions can be executed.
  • If the power supply voltage drops to low while data is written to EEPROM, the data may get corrupted. To mitigate this risk is recommended the use of BOD (Brown Out Detector) which can be enabled using the AVR's fuse bits.

Source: https://www.nongnu.org/avr-libc/user-manual/group__avr__eeprom.html