Mbed3 starting libwebsockets port

November 3, 2015

Mbed3 as a FOSS project

Right now mbed3 is in a bit of a strange place as a FOSS project. I guess this is a temporary situation but today a couple of things should be understood.

Choice of Apache for core code

First it's surprising to me the core stuff is Apache-licensed. For those, it seems to me it shouldn't be contraversial that fixes and improvements should be given back, since those parts are basically a commons used by all the vendors to provide mbed support. I would have expected the licensing to reflect that. On libwebsockets, I use LGPL2.1 with a Static Linking Exception, which would seem to be a good fit: it's explicitly not viral but it's explicit that that core library code wants your changes.

Other things like examples or that are expected to be built on project by project, sure Apache makes sense, they are there to enable individual usage that can perfectly well be proprietary.

Anyway it's not critical from user perspective, but it may negatively affect the amount and quality of contributions / fixes / discussion on the core code.

Contributions cannot be accepted due to CLA not ready

I sent four separate pull requests fixing broken build and other bugs and code improvements like remove needless include files. I got a variety of responses from "we already fixed it" (but they did not push to the registry...), we will take your patch "later" and finally the truth, which was they want to have a Contributor License Agreement, Ubuntu-style in place before accepting any patches.

So they are on github with all that entails, turning away fixes, because their Apache licensed project doesn't have a Contributor contract basically, which they rather optimistically think I would sign up to. Actually myself and a lot of other people do not like Ubuntu's CLA enforcement and won't sign up on principle.

There is definitely a thing for a while now you can call "github culture", many developers subscribe to a philosophy of fork-and-send-pull-requests specifically on github very casually as they find problems. Plugging into that is a big part of why you would put your project on github in the first place. These are a spread of developers, some of them top-notch, with a current - as in this second - interest in your project that are willing to take time to improve it for you and other users, immediately and unpaid. This can be a huge advantage sometimes dwarfing the internal developer effort.

I guess the people who want the CLA are thinking about maintaining a clear copyright on the code, for, eg, relicensing later. But... it's Apache licensed already. ARM can just fork it and relicense according to Apache terms already. Yes they don't have standing to fight copyright battles for code that came from elsewhere, but as of today all the core guts has come from internal devs, it's not like their code will disappear. And it's Apache licensed, what copyright battles are they expecting?

Anyway let's hope they rethink the CLA thing entirely, or if not, clearly state in their README.mds that contributions involve a CLA (which will turn a lot of people right off).

lws porting

So... the first move was modify the tcp echo sample application to be a very dumb http server itself... that worked fine. I then cleaned it and changed the style to kernel style since it is easier on my eyes. For reasons that will be explained, you can find this trivial conversion on github

https://github.com/lws-team/mbed3-dumb-http-test

That represents the guts of the "mbed3 way" for tcp networking, so I sat and stared at it for quite a while trying to figure out how to apply this in the posix-based libwebsockets code. The first choice was whether to abandon trying to use lws as it is, a CMake project that works on Linux, Windows, OSX, and many other (Posix) platforms and just rip code out of it to build on the dumb webserver until it worked.

I seriously considered that, but there are already many synergies with the existing library. Internally, mbed is using Cmake, lws uses it; mbed is event-driven, so is lws; mbed needs nonblocking, so does lws. And there is just a huge investment in the existing library code over 5 years now, with nearly 100 different contributors. It's better if the end result of this is increase the value of lws rather than fork off into some mbed-specific project that is no longer in sync. And there are many features and code paths in lws that are well-debugged as they are I can get the advantage of.

So I decided to go down the road of porting the library to mbed rather than mine pieces out of it, even though that is going to be a lot more work in the short term.

yotta project structure

Yotta's strategy is to have a top level "app" who has dependencies inside a generated ./yotta_modules subdir. For development, in git, that means your toplevel app is one git project, excluding ./yotta_modules, and in the toplevel ./yotta_modules, the libraries you are working on have their own git repos (also excluding their ./yotta_modules).

./lws-test-server
   ./yotta_modules
     ./websockets
       CMakeLists.txt

Currently lws contains his own test apps, but that can't fly with Yotta. So the structure is a new test server project, which has lws as a dependency, and develop lws at the same time as a git repo in the dependency directory. It implies if you have n test apps, you need n new toplevel git projects, and symlink to the library git repo for development, but that's a problem for another day since test-server is all I care about at the moment.

yotta CMake support

Yotta is evidently designed to work well with "modules" (libraries) as dependencies that themselves use CMake, it seems to label them "existing" modules.

When I built the stub toplevel test app, Yotta understood he was dealing with an "existing" project and built it well immediately. Unfortunately, since it's a mature CMake project that builds on many platforms, that won't work without some CMake option configuration. I looked for how to control that from the toplevel app, but there doesn't seem to be a way. I asked about it

https://github.com/ARMmbed/yotta/issues/556

(Edit: I got a very good reply when the UK woke up suggesting putting the mbed3 specific set()s in the generic library CMakeLists.txt conditional on YOTTA_<modulename>_VERSION_STRING existing... this worked great and I was able to remove my hacks forcing the default states at the definition).

Then it built, but with a buttload of warnings and errors.

mbed3 default compiler options

Libwebsockets has -Wall -Werror in his configuration for Linux by default, so I was surprised to see such a spew coming.

It turns out mbed3 default C[XX]FLAGS for gcc has -Wextra, which while I am a fan of listening to the compiler, brings in useless warnings like unused function parameter. There may be very good reasons why a function has a fixed set of parameters that under some conditions like preprocessor options, don't get referenced. At any rate it does not directly imply a problem, and the only solution is fill the code with (void)param; fake references to keep the compiler happy. -Wall already has the more useful unused variable.

Another very annoying useless -Wextra warning is because the code takes advantage of the sparse C99 array initialization syntax while leaving the other members at 0, it complains we did not elaborate every member in the initializer... for each missing member.

I gritted my teeth and "fixed" them in the code though, because it did also enable a useful warning about comparisons between signed and unsigned.

Still it would be cool if the yotta project definition JSON had a way to add / remove flags on the default compiler options as overrides on the target definitions Yocto-style, so we can snip out the warnings of dubious worth individually.

lws integration

After that it built... lws has a "platform" concept already, with mutually exclusive platform files that get built according to Windows, unix etc. I added lws-plat-mbed3.c and also .cpp that get built for mbed3 platform, and moves the networking class stuff into there from the dumb http server test app.

So after flashing the K64F, he could work as the dumb test app, and he was built against the library, although not using it.

After that, I moved the dumb test app's class into the library, and his member functions to lws-plat-mbed3.cpp, and adapted the toplevel test application to start the library.

That kinda worked...

IP: 192.168.2.205:80
4: Initial logging level 255
4: Libwebsockets version: 1.5 2e89c3c
4:  per-conn mem: 104 + 2108 headers + protocol rx buf
4:  Listening on port 7681
4:   Protocol: http-only
callback_http: reason 27
onIncoming
callback_http: reason 17
callback_http: reason 29
4: mbed3_tcp_stream_accept
callback_http: reason 19
4: onDisconnect

He properly starts the library, the logging is working, the callbacks are coming for the http protocol handler in the library when we connect. But he never sees the actual RX payload from the browser any more.

After puzzling about what changed, I eventually realized nothing changed but we take a bit longer after the socket accept before returning to the event loop. So I checked with telnet on port 80, where he connects but doesn't send anything until I type something, and that RX packet was processed properly.

IP: 192.168.2.205:80
4: Initial logging level 255
4: Libwebsockets version: 1.5 2e89c3c
4:  per-conn mem: 104 + 2108 headers + protocol rx buf
4:  Listening on port 7681
4:   Protocol: http-only
callback_http: reason 27
onIncoming
callback_http: reason 17
callback_http: reason 29
4: mbed3_tcp_stream_accept
callback_http: reason 19
4: onRX
3: x
4: onSent

The corresponding telnet session is like this, which is correct

$ telnet 192.168.2.205 7681
Trying 192.168.2.205...
Connected to 192.168.2.205.
Escape character is '^]'.
x
HTTP/1.1 200 OK

Ahaha... hello

I made a github issue report about it, and as part of that confirmed using the dumb test app this was not related to library integration, RX packets are lost if there is even a small delay after the accept. If the RX packet send from the client is delayed in time compared to the accept, as in telnet, there is no problem.

https://github.com/ARMmbed/sockets/issues/35

What we learned this time