Grounded clouds and post-breeze

Eating lunch in a cockpit greenhouse, napkin tucked in the bottom shirt between sweat and skin, the building in front of me is Google HQ; I’m an engineer; I’m on my way.

It’s not uncomfortable. It’s simply what I do. I have not the thought of displeasure in the proceeding; though I desire greater recompense, it is only fleeting. What stays is the idea that I must share it, and even summer winds can cool the self-drenched. How be it to live as a solo solitaire? To have as thoughts the empty, to be in the present, to gaze o’er a forest of cars and metal birds.

I downloaded MinGW. I had this obsessive ambition to create a Pelles C makefile to build a static libcurl, but felt I had spent enough time reading without writing. It was time to follow a recipe and to click icons. VBA lets you hand-wave a lot of things; in the interest of getting things done, sometimes compromise is necessary.

Yesterday I read about Javascript closures; tonight I shall do the same. Currently, I think of them as a way to create objects. Functions are first-class entities in Javascript: you can pass them around like variables. You can define a function within a function, and that inner function knows about the variables declared outside it. I’m still not sure whether to trust them. Are they solely an abstraction, or can mathematicians get away with declaring anonymous conjectures?

I figure I will “start using them” as a matter of course: no errors will occur except subtle ones, likely hair-pulling ones – those are my favorite, because they force me to slow down and analyze each step. Sometimes, the harder way is the better one.

Edit: There is such a thing as anonymity in mathematics. It is called “lambda calculus.”

Crash course in everything web

In college, web apps class was HTML and CSS, and Javascript was “an interesting graphics library.” Now it is used server-side, and I am beginning to appreciate its language conventions. I never thought the first instance of my entry into cloud development would be through a free host, a free app host, with free libraries, and learning from free lectures from one of the top CS programs in the country. But there it is.

My assumption is that Javascript still communicates through HTTP, so I don’t know if there’s potential for automation yet. Any request to the server for processing can still be done with cURL. My guess is JS is for interactive experiences and responsive web pages. It’s the “application language” of the web. But what can we accomplish with a headless WebKit?

The whole reason I used ssconvert was to avoid Excel VBA: you convert the workbook and munge the data. No user interface required. If we elevate the abstractions just to get a fancy AJAX workbook, I’m not sure that’s the direction to be going.

Set up a wireless USB adapter with a hostname.if(5) file

OpenBSD 5.3 detected a NETGEAR USB wireless adapter and labeled the interface urtw0. I created a /etc/hostname.urtw0 file:

nwid <ssid>
wpakey '<wpa>'
dhcp

After a reboot, I could visit web pages.

You can use vi to create the text file:

  • vi <file> to create the file
  • i to enter insert mode and begin typing
  • ESC to exit insert mode
  • :wq to “write and quit,” to save the file

Getting started with SQLite3 using Pelles C

Here is a basic overview of our steps:

  1. Download and install Pelles C.
  2. Download SQLite3 files.
  3. Write some programs and extract some files.

By the end of this tutorial, we will have the following files:

  • run-pocc.bat: a batch script to execute the Pelles C compiler, pocc. This compiles our C source files and produces .obj (object) files.
  • run-polink.bat: a batch script to execute the Pelles C linker, polink. This produces our executable from the object files.
  • sqlite3.lib: the extracted file from sqlite3.dll, which we get using the Pelles C polib tool.
  • main.c: sample code from http://milky.manishsinha.net/2009/03/30/sqlite-with-c/
  • sqlite3.c: from the SQLite3 amalgamation file
  • sqlite3.h from the SQLite3 amalgamation file

You will put all these files into a folder somewhere. Make this folder now.

You can download Pelles C here: http://www.smorgasbordet.com/pellesc/

You will need the SQLite amalgamation file and the SQLite DLL. For our purposes, we can use the DLL that comes with Pelles C. We need the .dll file to get the .lib file. The linker needs the .lib file.

You can download SQLite here: http://www.sqlite.org/download.html

Our next step is to set up Pelles C.

  1. Install Pelles C.
  2. Set up your PATH.
  3. Test your tools.

For this tutorial, I used the 32-bit version of the Pelles C IDE. The download comes with tools in the Bin\ directory. Add an environment variable, PELLESBIN, with the value C:\Program Files\PellesC\Bin, and then add %PELLESBIN% to the end of the Path environment variable.

Open Command Prompt and type pocc. It should bring up the Pelles C compiler options.

Next, we will grab the amalgamation file from the SQLite site. Copy the C source file and header file to the folder you made. Also copy sqlite3.dll from the PellesC\ directory to your folder.

Ok, now run

> polib sqlite3.dll /OUT:sqlite3.lib

We’ll mention this .lib file to the linker with the /DEFAULTLIB option later.

Create a file main.c and copy-paste manishsinha’s code into it. This will be our sample program to test that SQLite has been successfully set up.

Execute run-pocc.bat. If there are no errors, execute run-polink.bat.

You should now have a test-sqlite.exe file that you can run from the command-line.

Now that you know how to do it “the hard way,” I leave it as an exercise to figure out how to make this work “the easy way” with the IDE.

Script links:

Fundamentals and where we lay our bets

The Coursera startup class has begun. We have a lot of articles to read and technologies to learn. I want to give it an honest shot, but here is a question:

Can we use Javascript to automate the web?

If you work eight hours a day – or more – interacting with a web form to glean information, and copy-pasting results back and forth between Excel spreadsheets, would you want more of the same, or a method to batch-process a whole list of things?

Do you guys count the number of keystrokes that add up with Alt-Tab, Ctrl-C and Ctrl-V? Double it because you’re switching contexts between browser and workbook. Add in the left clicks and the waiting for the HTTP response, the browser to render, and so on.

I bet that doesn’t enter your mind though. You’re all about user interface (UX) and metrics. And making money. These are all good things, but they are not better things. We are trending toward ubiquitous Javascript and tasks centralized in the browser.

What are you doing about automation and reducing RSI?

Enough layers to make it

I wrote a preliminary Win32 GUI today. There was nothing I couldn’t look up, and there were almost no compiler errors. I’m impressed with how well the Windows API abstracts C into a collection of function calls that make the code look like Java Swing or AutoIt. It was really neat.

That leaves the model, the business logic. That’s where the rubber meets the road, where pointers and heaps and the rest collide, where I earn my keep. In truth, my work is sort of a port: not original in the least, but a vector for learning.

I am inspired by ESR’s work on GPS software. His team was able to write C (and Python) code with tests and simulations to demonstrate the program’s correctness. That is the rigor which I would like to apply to C programming.

  • GPSD [Architecture of Open Source Software, Vol. 2]

Low-level trafficker

Today I answered the question of how to work on C at work: learn the Win32 API.

You’ll see on my header a link to “OpenBSD,” and I willingly put on these oil-covered gloves – even DWORDs, PWSTRs and the rest of that all caps-blasted library has a silver lining. I’ll take it, even if it means a little more Windows.

For C is an “expressive” language with a handful of concepts, according to K&R, and its beauty may still be realized amid this compromised forest. Eight hours of writing C, friends: not even Handel, fettered in solitude away from the city, could reach out and take in a fraction of my resources. I am only a search away from every bit-byte, malloc() concept to scramble the brains of men.

I embrace it willingly, even so the GUI – the “windows” of Windows – with yet sternness of an imperfect being: hopefully I will not need MFC, even as I wonder how I can leave this rabbit hole.

A web stack library

I want to use source control as it was originally designed: for collaboration. I want my fellow IT peeps to bone up on programming and contribute back to a company-wide toolset. Somehow, we’ll make a series of utilities that people “on the ground” can employ in their tasks; the code culture will grow from both soil and sky.

The question is how does one take a bunch of cURL scripts and package them into an API? They’re already useful on their own. I think I will have to use libcurl in some fashion. The connection framework will need to be bulletproof, though. Empty replies, no replies and timeouts need to be handled gracefully.

Compounding this difficulty is OS architecture. I really want to understand POSIX concepts, but Windows “is just there.” Somehow, I’ve got to be able to work on C for eight hours a day; that would be awesome. Painful, I hear, but awesome.

I work on my weaknesses. Isn’t it counter-intuitive? But I want to be good at math.

Customer requirements

I rolled up a TCP client-server and put out the alpha. I searched “single-threaded server” and thought about the I/O-blocking horror. If we set up all the machines beforehand, we could avoid that bottleneck. I thought about fast array lookups and cached indexing, logarithmic runtimes and great success.

“You shouldn’t make a form without first finding out what we want it to look like and what we want out of it,” the manager said.

So much for letting loose. We were explorers in a warehouse jungle, girder-shelves, cardboard trees. Our connections were spotty; we kept the control file in our electronic bibles and kept the scanners unsheathed.

I thought about our reporting process. We can’t do realtime. But we can do realtime extrapolation. If I can retrieve the data in 15-minute intervals, process it up in the between latent, run a linear regression and spit out a .png from R, and then mail it – bam: steaming-fresh, friendly, twice-daily works of art!

Enterprise programmer needed

I’ve been pretty productive at work with scripts and macros. I wonder what kinds of problems require Java and custom data structures & algorithms. Do those companies develop their big solutions in-house, or are C#, Java et al. overkill for extending existing licensed products?

My ignorance is partially a blessing, because I get to scale with the company. If indeed there were requirements for concurrency and full-stack development, I would start hitting upon the wishlists of many recruiter-oriented job openings. And on the other side is my own personal skill wishlist: C, UNIX tools, machine learning and statistics.

It’s great to know a bunch of technologies, but if I could learn anything it would be math and computer science. Oh, and IPC. I would love to master that: sockets, pipes, shared memory.