Embedding Bootstarp into Microcontroller

S

SBSP

Senior Member
Joined
Sep 7, 2007
Messages
667
Reaction score
16
Hey All!

Need some advice please. I’m trying to embed bootstrap into a wireless microcontroller (ESP8266)
I already managed to get it working but making use of a CDN instead of local CSS. I will not need the bootstrap Java script.
But ideally it must be available locally other wise the device cannot be used when the client does not have internet.

The ESP module has 4MB of flash space available and its chip has plenty power to handle a small single client web server.

The Challenge.
I have to put the CSS and HTML code into a string and then write the string to the ESP8266 web server output function.
I cannot simply place the .min.css version of bootstrap into one long line. I will get a string space related error.

I am however able to break it up in chunks and then send it in pieces (VERY MESSY CODE). I would like to approach it with a slightly more clever way.

I want to try use my own kind of compression. Base64 seems like a bad idea (If I even may call it compression) other types of compression is just plain confusing for me to implement.
So my Idea is to create an index of special characters that goes beyond ASCII (127) . And Assign those to commonly used HTML and CSS tags.
That way I can for instance replace things like <body with a special character. Or even pack string numbers larger than 3 digits Then only when I’m writing to the Web server output function do I “decompress” it.

How would you go about it. ?
 
Hmm had the same need. Makes it easy to create a nice and powerful UI.
Your idea is quite good. That is more or less how ZIP works (dictionary). But is using an SD card not easier? You can hook up SD card to ESP e.g. http://www.instructables.com/id/SD-Card-Module-With-ESP8266/
and then serve from there. Makes the dev effort also way simpler when you need to make changes etc. Depends on your security requirements though. You could serve the js/css file for TWB and jquery from card (its publicly avail anyway) and then have your html/js and css in flash if you want to protect it. But personally I would not bother.
 
Thanks. The reason I want to stay away from an external SD card is because it will complicate things for the ones who are not so familiar with the whole Arduino thing. It will be made open source and I want it to be as simple and fail safe as possible.

The ESP will behave as a Wifi to Serial transceiver that will be connected to any Marlin Based 3D printer. It will allow you to wirlessly upload G-code to the 3D printer's SD. So if I had to go with the external SD to load stuff on another SD It gets confusing.

Who ever wish to install this only needs to connect TX,RX of the ESP to the Printer's Serial RX and TX lines and that's it. Seeing I'm making use of the NodeMCU it can be powered with 5V without worrying about voltage regulators.

Its far from complete.
Untitled.png
2.png

I want people to be able to simply write the binary of the sketch directly to their NodeMCU and then use putty to configure the login details as shown in the picture.
 
SBSP said:
I stumbled onto something.

https://circuits4you.com/2018/01/31/example-of-esp8266-flash-file-system-spiffs/

You can store large text files in part of the flash memory using a file system using a library.
Can imagine if this is used carelessly the stuff you store can easily be forgotten.
Click to expand...

Yes i remember this now. As long as the files are small enough tho.
That web page looks moer nice. Did that come from esp or is that how you intend it to come from esp? U should also consider using websocket or mqtt to push realtime updates to the page w/o having to refresh. I doubt that if you use SD card that users will get confused. If you place the sd card for files on the pcb close to esp inside the enclosure and then the sd card slot for the printer files such that it is accessible from out the enclosure then you wont have ana issue? Hell you can even ship the printer file to print an enclosure on it as well.
 
Spacerat said:
Yes i remember this now. As long as the files are small enough tho.
That web page looks moer nice. Did that come from esp or is that how you intend it to come from esp? U should also consider using websocket or mqtt to push realtime updates to the page w/o having to refresh. I doubt that if you use SD card that users will get confused. If you place the sd card for files on the pcb close to esp inside the enclosure and then the sd card slot for the printer files such that it is accessible from out the enclosure then you wont have ana issue? Hell you can even ship the printer file to print an enclosure on it as well.
Click to expand...

I will look into the SD card option maybe it will be better. I have a couple of readers somewhere in a box. I wont be shipping anything though. People must build their own. I will however make the enclosure available on thingyverse. Its just a fun project not looking to make bucks out of it :)

And yes the interface as shown in the Picture comes from the ESP, The temperature values on there
are off coarse not real. I still need to hook ESP8266 up to a Arduino Mega with Marlin loaded onto it.
I just hope it will run without having a RAMPS connected to it.

Johnatan56 said:
Since you're not using a lot of bootstrap, maybe trimming it?
Use Chrome's built-in tool to find unused css.
https://developers.google.com/web/updates/2017/04/devtools-release-notes#coverage
Click to expand...

I thought of stripping it down but then I wasn't sure how I was going to find the elements. Never thougt of using chrome to find them.
 
Last edited:
Its coming along nicely. I haven't figured out the bootstrap embedding part yet. But I WILL find a way :-)

At this sage it it can successfully communicate with Marlin running on an Arduino Mega with no RAMPS connected to it.



To prevent page refresh It's using Java script HTML requests to fetch data from it self on a internal mini web service link.
I does not like it when you try to fetch different data streams at the same time, it falls over and reboot's :-)
So I have to take turns, which creates a bit of a delay. But not all information such as the printer's name is critical.

The below is live data running on the actual NodeMCU . I'm going to leave it overnight to see if it will crash or not.

Wirlessly.jpg

Next is to get the File Progress and GCode upload to work correctly.

I haven't done much JS in the past and I have never really bothered with JS DOM to change elements.

when I enable the debugger in Firefox (Shift+F5) I noticed
XMLHttpRequest has been depreciated ? What should I be using instead ?
Kind of a bummer I only realized this after most of the work was done.

PHP: 
function() 
	{

		var ConnectStatus="Status: Idle";
		document.getElementById("connectstatus").innerHTML = ConnectStatus ;
		if(FetchFirmware==1)
		{
		var url1 = "/retval.php";
		var xhttp = new XMLHttpRequest();
		xhttp.onreadystatechange = function() {
		if (this.readyState == 4 && this.status == 200) {
       

		var E1B1TempVal = xhttp.responseText ;
		if (E1B1TempVal.substring(0,5)=="ok T:")
		{	
			ConnectStatus="Status: Fetching";
			document.getElementById("connectstatus").innerHTML = ConnectStatus ;
			document.getElementById("demo").innerHTML = FetchFirmware ; 
			var E1TempVals = E1B1TempVal.substring(E1B1TempVal.indexOf("T:") + 2, E1B1TempVal.indexOf("B:"));
			var E1Vals = E1TempVals.split("/", 2); 
			var E1Current = E1Vals[0].trim(); 
			var E1Target = E1Vals[1].trim(); 
		}else{E1Target="1";E1Current="1";}	
		
		var B1E1TempVal = xhttp.responseText ;
		if (B1E1TempVal.substring(0,5)=="ok T:")
		{			
			var B1TempVals = B1E1TempVal.substring(B1E1TempVal.indexOf("B:") + 2, B1E1TempVal.indexOf("@:"));
			var B1Vals = B1TempVals.split("/", 2); 
			var B1Current = B1Vals[0].trim(); 
			var B1Target = B1Vals[1].trim(); 
		}else{B1Target="1";B1Current="1";}	
	   var E1percent= E1Current.trim()/E1Target.trim()*100;
	   var B1percent= B1Current.trim()/B1Target.trim()*100;
	   document.getElementById("ProgressE1Temp").setAttribute("style", "width: " + E1percent + "%"); 	
	   document.getElementById("ProgressBedTemp").setAttribute("style", "width: " + B1percent + "%"); 	   
       document.getElementById("ExtruderTemp").innerHTML = E1Current + "&deg;C";
	   document.getElementById("ExtruderTargetTemp").innerHTML = E1Target + "&deg;C";

	   document.getElementById("BedTemp").innerHTML = B1Current + "&deg;C";
	   document.getElementById("BedTargetTemp").innerHTML = B1Target + "&deg;C";	
	   document.getElementById("debug").innerHTML = "<small>" + E1B1TempVal + "</small>" ;
	   
		}
				
		};		
		xhttp.open("GET", "/retval.php", false);
		xhttp.send();
		}
		
		if(FetchFirmware==0)
		{
		var url1 = "/retinfo.php";
		var xhttp = new XMLHttpRequest();
		xhttp.onreadystatechange = function() {
		if (this.readyState == 4 && this.status == 200) 
		{
		var GenInfo=xhttp.responseText;
		if (GenInfo.substring(0,8)=="FIRMWARE")
		{
		FetchFirmware=1;
		var Firmware=GenInfo.substring(GenInfo.indexOf("FIRMWARE_NAME:") + 14, GenInfo.indexOf("PROTOCOL_VERSION"));
		var PrinterName=GenInfo.substring(GenInfo.indexOf("MACHINE_TYPE:") + 13, GenInfo.indexOf("EXTRUDER_COUNT"));
		var UUID=GenInfo.substring(GenInfo.indexOf("UUID:") + 5, GenInfo.indexOf("Cap"));
		document.getElementById("printername").innerHTML = PrinterName ;  
		document.getElementById("firmware").innerHTML = "<small>Firmware: " + Firmware  + "</small>";  
		document.getElementById("uuid").innerHTML = "<small>Serial Number: " + UUID  + "</small>";  
		document.getElementById("debug").innerHTML = "<small>" + GenInfo + "</small>" ;
		}	      
		}
		};
		xhttp.open("GET", "/retinfo.php", false);
		xhttp.send();	
		}	

	}
 
Last edited:
:mad: I officially gave up on the ESP8266.

It simply does not like being a web server and a serial client at the same time. After endless crashes I tested by starting a new very basic project where it would fetch serial data from another Arduino. It randomly reboots. I can also expedite it by by increasing the serial read rate and by refreshing the browser by holding down F5. If I leave it to send Serial to the serial port every 10 seconds it would run for about 10 mins then it will crash and reboot.

I even tried another NodeMCU and it does exactly the same. So either the hardware is not up to it or they Web server libraries has a problem.

I personally think the hardware is not robust enough. It's weird because In the past i managed stream youtube via it to my phone.

I'm going to give a go at the ESP32 and to what it does.

Looks like this guy has the exact same "Idea" and problem as me. :D
https://github.com/esp8266/Arduino/issues/428
 
Last edited:
Your code might have a memory leak. I have also run http server as well as serial with no issues
 
Spacerat said:
Your code might have a memory leak. I have also run http server as well as serial with no issues
Click to expand...

I think I , know what it is. USB only provide 500mA if i remember correctly. Often i read that the ESP8266 should not be powered from USB. I tried it on a Samsung cellphone charger and it seems like it's working a bit better. But don't have enough time to test.
 
SBSP said:
:mad: I officially gave up on the ESP8266.

It simply does not like being a web server and a serial client at the same time. After endless crashes I tested by starting a new very basic project where it would fetch serial data from another Arduino. It randomly reboots. I can also expedite it by by increasing the serial read rate and by refreshing the browser by holding down F5. If I leave it to send Serial to the serial port every 10 seconds it would run for about 10 mins then it will crash and reboot.

I even tried another NodeMCU and it does exactly the same. So either the hardware is not up to it or they Web server libraries has a problem.

I personally think the hardware is not robust enough. It's weird because In the past i managed stream youtube via it to my phone.

I'm going to give a go at the ESP32 and to what it does.

Looks like this guy has the exact same "Idea" and problem as me. :D
https://github.com/esp8266/Arduino/issues/428
Click to expand...

Have you looked at the ESP32?

Edit: duh... sorry did not read the full post..
 
SBSP said:
Looks like this guy has the exact same "Idea" and problem as me. :D
https://github.com/esp8266/Arduino/issues/428
Click to expand...

That problem was solved in 2015 tho.

Also to serve static files: https://github.com/esp8266/Arduino/...266WebServer/examples/FSBrowser/FSBrowser.ino

You can extract the part about serving those files from that example. The EPS8266 code also includes a util to upload data to the flash. When you compile you can decide your ratio of flash/program space (typically 1mb program/3mb storage)
 
I finally found the issue.

The void() function
PHP: 
      if (incomingByte != 13)
                {
                 if(incomingByte != 127){cmdBuild = cmdBuild + incomingChar; }else{cmdBuild= cmdBuild.substring(0,cmdBuild.length()-1);}//Remove Backspaces and the last mistake chracter
                }
                else
                {
                   if(WebClientMode==0)
                  {
                  Serial.println("");
                  if (cmdBuild=="help"){Printhelp();CommandValid=1;}
                  if (cmdBuild=="dump config"){Printdumpconfig();CommandValid=1;}
                  if (cmdBuild=="clear config"){Printclearconfig();CommandValid=1;}
                  if (cmdBuild=="save config"){Printsaveconfig();CommandValid=1;}
                  if (cmdBuild=="welcome"){Printwelcome();CommandValid=1;}
                  if (cmdBuild=="reboot"){Printespreset();CommandValid=1;}
                  if (cmdBuild=="about"){Printabout();CommandValid=1;}
                  if (cmdBuild=="reload config"){loadconfig();CommandValid=1;Serial.println("Config reloaded.");}
                  if (cmdBuild=="clear"){clearAndHome();CommandValid=1;}                                
                  if (cmdBuild.substring(0,11)=="set apssid "){Printsetssid();CommandValid=1;}
                  if (cmdBuild.substring(0,11)=="set appass "){Printsetpass();CommandValid=1;}
                  if (cmdBuild.substring(0,14)=="set adminuser "){PrintSetAdminUser();CommandValid=1;}
                  if (cmdBuild.substring(0,14)=="set adminpass "){PrintSetAdminPass();CommandValid=1;}
                  if (cmdBuild.substring(0,16)=="set refreshrate "){SetRefreshRate();CommandValid=1;}
                  if (cmdBuild.substring(0,13)=="set baudrate "){Printsetbaudrate();CommandValid=1;}
                  if (cmdBuild.substring(0,5)=="send "){SendGcode();CommandValid=1;} 
                  if (cmdBuild.substring(0,10)=="testserial"){TestSerial();CommandValid=1;} 
                  if (cmdBuild.substring(0,9)=="scan wifi"){ScanWifi();CommandValid=1;}   
                  //If command is invalid in non-webclientmode
                  if(CommandValid==0){Serial.println("'"  + cmdBuild + "' is not recognized as an internal or external command.");}
                  }

It reads the incoming serial data and it was too taxing the CPU. If the web server takes to long to serve the html code the internal watchdog times it out and resets the ESP.

So I created a secondary void function for reading serial, "Setup Mode" for when serial data comes in from putty. And Web client mode which kicks in once you log in via the web front end.

Its now stable!
So we learn :)
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X