Friday, November 4, 2016

Optimizing C++ android code for minimum size

Any more or less serious Android  C++ code (NDK) will immediately bring a huge piece of C++ stdlib into your native library, something like 150kb+ surplus per ABI (cpu arhitecture).
Even such as this:
char *data = new char[1000];
More funny example is having a huge complex C++ class defined (several kloc including several subclases), which is producing like 30kb binary when stack-instantiated ("MyClass obj;"), and like 200kb binary when created using "new" operator ("MyClass *obj = new MyClass()").

Long story short this all happens due to a huge amount of exception handling code (which invokes iostream and other bullshit)  linked to your code due to just a single "new" operator (to handle bad_alloc exceptions), and yes, -fno-exceptions won't help you at all.

Solution is to link against minimal system STL (the only one built with no exceptions code for new/delete), by modifying your application.mk (if you are using plain NDK build) with the following line (there are a lot examples on the net):
APP_STL := system
I wasn't able to find ready solution for the modern Gradle (Android Studio) + CMake case, so had to spend a bit of time studying this fairly bad documented stuff. Finally, here we go, a piece of config you came here for (place it into your module gradle file) undo "android.defaultConfig.externalNativeBuild.cmake" element:
arguments "-DANDROID_STL=system", "-DCMAKE_CXX_STANDARD_LIBRARIES=-Lm"
Technically a ADROID_STL parameter definition should be enough, but I was still getting a rogue
"cxx-stl\gnu-libstdc++\4.9\libs\...\libsupc++.a" library sneaking into my linking command, so standard libraries override came to fix.

While we are here:
set(CMAKE_CXX_VISIBILITY_PRESET hidden)
set(CMAKE_C_VISIBILITY_PRESET hidden)
set_property(TARGET native-lib PROPERTY INTERPROCEDURAL_OPTIMIZATION True)
 Adding these lines to your CMakeLists.txt should hide your exports not marked with JNIEXPORT, and MAY BE enable link-time-compilation/optimization for some ABIs (third line, notice library name argument). Might save you a few KBs as well (hiding non marked exports is a good thing anyway).

Tuesday, May 3, 2016

Limiting MongoDB memory usage with cgroups on Ubuntu 14.04

MongoDB is memory mapping database files, so you Free RAM won't last long. It might be fine if you believe in Linux Memory Management, however I feel quite anxious when mongod takes every single bit of 512Mbytes of RAM of my small DigitalOcean (3 month 1core/512M/20G VPS for 5$, yes 5 bucks) box. Yes I'm chip and I don't want my nginx or nodejs to starve.

Ok, so there is a solution to this - cgroups. Unfortunately there are no config templates with cgroups packages (at least for Ubuntu 14.04), this makes the topic quite obscure. I written some small bash script which will ask you for desired MongoDB memory limit, create configs and init scripts and restart services.
IMPORTANT. This shell script will overwrite all your existing cgroup configuration and init files, please make sure you have nothing to loose.

sudo bash -c 'curl -o- http://brains.by/misc/mongodb_memory_limit_ubuntu1404.sh | bash'

You can find a good write-up on topic here, but it doesn't contain init scripts, so your changes won't last a reboot: https://www.percona.com/blog/2015/07/01/using-cgroups-to-limit-mysql-and-mongodb-memory-usage/

A good page on cgroups with Ubuntu 14.04 is here, but it contains no MongoDB specifics:
http://frank2.net/cgroups-ubuntu-14-04/


Friday, March 4, 2016

Creating delegate declared as internal in thirdparty library

Creating delegate declared as internal in thirdparty library (like mscorlib.dll) can be tricky, especially if it takes or returns arguments (classes) also declared as internal. It is pretty easy to use prepared delegate value to replace it somewhere in library internals using reflection, but attaining such delegate for your own method is completely different beast.

Lets look for example at SocketHandler delegate (mscorlib.dll):

internal delegate SocketHandler SocketHandlerFactory(
  Socket socket,
  SocketCache socketCache,
  String machineAndPort);

Beside the internal delegate itself (which we can't reference), we also have issue with SocketCache argument and SocketHandler return types both declared as internal (so we can't declare method to suit this delegate).
Long story short, the only way I found to achieve this is using DynamicMethod class which allows to declare a new method, with arguments we get from original delegate using reflection, emit some IL code to invoke our hardcoded method with untyped (object) arguments, and create delegate based on provided type.
Below there are some cuts from source code which should give you the general idea (yeah, I'm to lazy to provide full working example or fix variables names for you). Let's say we have object of SocketCache type (which has _handlerFactory delegate field we want to replace):


var sct = socketCache.GetType();
// get field info
var miHandlerFactory = sct.GetField("_handlerFactory"
  BindingFlags.NonPublic | BindingFlags.Instance);
// get variable original value  (delegate)
var oldHandlerFactory = miHandlerFactory.GetValue(socketCache);
// get type of delegate
var hft = oldHandlerFactory.GetType();
// get MethodInfo for delegate method
var hfmi = hft.GetProperty("Method").GetGetMethod()
  .Invoke(oldHandlerFactory, null) as MethodInfo;

// we will create delegate bound to our SocketFactoryInterceptor object
var sfit = typeof (SocketFactoryInterceptor);

// create dynamic method with our object as owner and a copy of all arguments of delegate
DynamicMethod dm = new DynamicMethod("Handler", hfmi.ReturnType,
  new []{sfit}.Concat(hfmi.GetParameters().Select(p=>p.ParameterType))
  .ToArray(), sfit, true);

var ilg = dm.GetILGenerator();
// load all provided arguments onto stack
foreach (var a in dm.GetParameters().Select((p,i)=>i)) 
  ilg.Emit(OpCodes.Ldarg, a);
// invoke our method (which can declare argument as simple untyped objects)
ilg.EmitCall(OpCodes.Call, sfit.GetMethod("CreateSocketHandler"), null);
// we need that cast for our predefined method to be able to return simple object type
ilg.Emit(OpCodes.Castclass, hfmi.ReturnType);
ilg.Emit(OpCodes.Ret);

// interceptor class will do all the stuff (call original handler and do our logic), so we just need
// to pass control to our existing method - as little IL as possible
var sfi = new SocketFactoryInterceptor() { transportSink = ns, 
  socketCache = socketCache, miHandlerFactory = miHandlerFactory, 
  handlerDelegate = (Delegate) oldHandlerFactory, dynamicMethod = dm, };
// create delegate based on delegate type information we got from variable
var dlg = sfi.ourDelegate = dm.CreateDelegate(hft, sfi);

// replace variable value with our new delegate
miHandlerFactory.SetValue(socketCache, dlg);