CodeGuru content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Introduction
A MySQL Storage engine requires functions that MySQL provides. Therefore, you normally need to link it into mysqld-core under Windows because mysqld.exe doesn’t export all required functions (it’s an .exe, not a .dll). So, you’re unable to import the required functions in your plugin.
At least in theory.
However, I noticed that the MySQL guys were kind enough to ship a .map file with their release. It contains all the addresses of the required import functions. So, I had the following idea:
Microsoft introduced the Delayloading feature with the Microsoft Visual C 6 linker. This feature enables you to load required DLL functions during runtime. To do that, the linker binds the IAT-Pointers to a special import code that basically does LoadLibrary() and GetProcAddress() on the first access to the required function and then overwrites the IAT-Entry with the correct address. For more information on delay loading, see: http://www.microsoft.com/msj/1298/hood/hood1298.aspx.
Fortunately, the linker doesn’t generate the delay-loading code himself. Instead, it just calls some pre-defined functions from delayimp.lib, which is shipped with Microsoft Visual Studio 6 and above. Microsoft fortunately also ships the source code for the delay import loader in the VC98IncludeDELAYHLP.CPP file. So, you already can guess what my idea is:
You need to write your own delay-Import loader that resolves the entry points to the requested functions in the .map file and writes the correct function pointer for it to the IAT. Because the plugin is running in the context of the mysqld.exe process that contains all the code, you just need a far pointer to the required code. Of course, you have to check whether the .exe file was relocated and eventually relocate the addresses from the .map file, but this shouldn’t be a big problem because the required load address is also in the .map file (address - required load address + real load address). Because the module handle of a module is identical to the load address, this is easy. So, you create your own delayimp.lib that you link with your plugin and resolve all imports dynamically during runtime. However, by design, the linker still needs an Import library for the function declarations to resolve.
Now, here you go…
First, you need to set up your build environment where you will do all the work. Create the windelayload directory in your MySQL Source directory.
Building the Import Library
1. Create a static import library
You have to build a library file out of all the objects from mysqld so that the linker is satisfied. To get a list of the objects that need to be used, I created a little batch script that collects from the build.make file cmake creates during the build process of mysqld. Because you need the compiled objects and the build.make file, you first need to compile mysqld, if you haven’t already. To do so, you can type:
cmake.exe" . -G "NMake Makefiles"
in the root directory of the MySQL Sources. Put your make_lib.cmd batch script in your working directory (windelayload).
Open a shell in this directory and set up your VC Environment because you need the lib.exe tool in the path to run the script properly (run vcvars32.bat). Then, run the script:
make_lib
This packs together all objects into a new static library: mysqld-test-static.lib. In fact, you could use the plugin now, after linking to this static .lib, because you now have all your required functions included. However, this is silly, because your .dll would be as big or even bigger as the mysqld.exe, so it doesn’t really make sense. So, in the next step you need to convert it to a DLL and produce an import library for the linker so that you can use the delayimport directive. You won’t be able to create a .dll, but creating an import library should work and this is enough.
2. Convert the static library to a DLL
The next step will be to convert the static library to a DLL. Because this is quite a complicated task (there is no possibility to create a DLL automatically that exports all symbols with the default MSVC Toolchain), you have to use a little helper script again. Part of the credits go to Mr. Grigore Stefan, who originally wrote a conversion script that can be found at http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=496762&SiteID=1.
However, I needed to adapt the script to use it for the MySQL import library creation. The original script was quite slow and didn’t create a valid .def file for data exports; it didn’t mark them so every exported symbol was a function for the import library, and this is wrong.
To create a proper .def file for the exports, you need the help of two GPL programs: the MinGW dlltool and the GNU Stream editor sed. Dlltool creates a valid .def file with all exports from the static lib you created before and the stream editor fixes a few errors in the created .def file. See the batch script for details. So, dlltool.exe and sed.exe need to be in the path. If you don’t have them, download them and extract them to your working directory. On your shell that has already been been set up with the Visual C build environment, run:
lib_to_dll mysqld
This could take quite some time, but if everything works it should result in the mysqld.lib import library.
Set Up Your Plugin
3. Add delayloader to your plugin and tell the linker to link dynamically
Next, you want to delayload the non-existant mysqld.dll with your delayloader code in your storage engine plugin so that the delayloader can do its job. Therefore, you have to tell the linker to use it. Because MySQL uses the Cmake build system, edit your CMakeLists.txt file accordingly. Add the following lines to it (and edit them accordingly):
SET(DELAY_LDR_DIR ../../win/delayload)
SET(DELAY_LOADER ${DELAY_LDR_DIR}/mysqld ${DELAY_LDR_DIR}/mapdelayldr)
SET_TARGET_PROPERTIES _
(your_storage_engine PROPERTIES LINK_FLAGS "/DELAYLOAD:mysqld.dll")
TARGET_LINK_LIBRARIES(your_storage_engine ${DELAY_LOADER})
A few words about the delayloader:
It reads the .map file in the directory of the hosting process (in your case, the mysqld.exe) into a hashtable. Then, it resolves the addresses using this table. However, there is a little difficulty with the delayloader module:
Unfortunately, there are differences in the VC6 and VC7+ linker for the delay loader because, when 64-bit architecture emerged, the designers of the delayloader discovered that they should have used RVAs instead of pointers in the ImgDelayDescr. Therefore, you have to handle both linker versions with the correct linking code. Each Visual Studio version has a different header file. The delayloader code tries to handle this and compile for the correct version. This delayloader was only tested with the VC6 linker; there is no guarantee that it will work with the new delayloader, but theoretically it should.
4. Link it
Next, you can try to build your makefiles with cmake and link it. I’m using nmake for this purpose, so I’m executing cmake . -G "NMake Makefiles" in the MySQL main directory; afterwards, I’ll build my plugin using the nmake command. But, it should also work using other cmake targets.
Depending on your plugin, you may get linker errors referring to unresolved external symbols. They usually refer to global data variables that are used by your plugins. For example, when linking my plugin initially, the linker showed the following unresolved symbols:
ha_storage.obj : error LNK2001: Nichtaufgeloestes externes Symbol _ "class Bitmap<64> const key_map_empty" _ (?key_map_empty@@3V?$Bitmap@$0EA@@@B) ha_storage.obj : error LNK2001: Nichtaufgeloestes externes Symbol _ _my_charset_bin ha_storage.obj : error LNK2001: Nichtaufgeloestes externes Symbol _ "struct charset_ info_st * system_charset_info" _ (?system_charset_info@@3PAUcharset_info_st@@A) storage.dll : fatal error LNK1120: 3 unaufgeloeste externe Verweise
