xmake v2.3.1 released, Seamless integration with other build systems

In the past two months, I have made a lot of refactorings to improve xmake and added a lot of useful new features. Welcome to try it.

Some new features:

Compile other projects maintained by the build system with one click to achieve seamless docking and support cross compilation (such as fast cross compilation of autotools, see details below)
Added xmake project -k ninja project generation plugin to support generation of build.ninja build system files

Some improvements:

Improve command line parameter input, support *nix style parameter input, thanks @OpportunityLiu for contribution
Improve tab command completion, add command completion support for parameter values
Optimize get.sh installation and xmake update update scripts, add domestic mirror source, speed up download and install updates
gcc/clang compilation error output support native color highlighting support
Added msys/cygwin platform, and xmake source code also supports msys/mingw platform compilation

Some invisible improvements:

Add socket and pipe modules and improve the process module
Refactor the whole process scheduler, better scheduling and parallel construction
Refactoring and improving the entire coroutine coroutine module, supporting simultaneous scheduling support for socket/pipe/process (preparing for subsequent remote compilation and distributed compilation)

There are also some scattered bug fixes, see updates below.

Introduction of new features

Generate build.ninja build file

xmake now supports the generation of ninja build files, allowing users to use ninja to quickly build projects maintained by xmake. I have to admit that in terms of build speed, ninja is indeed much faster than xmake. I will try to optimize the build speed of xmake in subsequent versions.

$ xmake project -k ninja

Then call ninja to build:

$ ninja

Or use the xmake command directly to call the ninja build, see below.

Try building with another build system

xmake v2.3.1 and above directly interface with other third-party build systems. Even if other projects do not use xmake.lua for maintenance, xmake can directly call other build tools to complete the compilation.

Then the user can directly use a third-party build tool to compile, so why use xmake to call it? The main benefits are:

Completely consistent behavior, simplifying the compilation process. No matter which other build system is used, you only need to execute the xmake command to compile. Users no longer need to study the different compilation processes of other tools
Docking the configuration environment of xmake config, reuse the platform detection and SDK environment detection of xmake, simplify the platform configuration
Docking cross-compilation environment, even for projects maintained with autotools, you can quickly cross-compile through xmake

Build systems currently supported:

autotools (cross-compiling environment for xmake)
xcodebuild
cmake
make
msbuild
scons
meson
bazel
ndkbuild
ninja

Automatically detect build system and compile

For example, for a project maintained using cmake, executing xmake directly in the project root directory will automatically trigger a detection mechanism, detect CMakeLists.txt, and then prompt the user if cmake is needed to continue compiling.

$ xmake 
note: CMakeLists.txt found, try building it (pass -y or --confirm=y/n/d to skip confirm)?
please input: y (y/n)
-- Symbol prefix:
-- Configuring done
-- Generating done
-- Build files have been written to:/Users/ruki/Downloads/libpng-1.6.35/build
[  7%] Built target png-fix-itxt
[ 21%] Built target genfiles
[ 81%] Built target png
[ 83%] Built target png_static
...
output to/Users/ruki/Downloads/libpng-1.6.35/build/artifacts
build ok!

Seamless using xmake command

Currently supports common commands such as xmake clean, xmake --rebuild and xmake config to seamlessly interface with third-party systems.

We can directly clean the compiled output files of the cmake maintenance project

$ xmake clean
$ xmake clean --all

If you bring --all to perform the cleanup, all files generated by autotools/cmake will be cleared, not only the object files.

The default xmake is docked with incremental build behavior, but we can also force a quick rebuild:

$ xmake --rebuild

Manually switch the specified build system

If there are multiple build systems under maintenance in a project, such as the libpng project, which comes with autotools/cmake/makefile and other build system maintenance, xmake defaults to using autotools by default. If you want to force switch to other build systems, you can execute:

$ xmake f --trybuild=[autotools|cmake|make|msbuild|..]
$ xmake

In addition, the --trybuild= parameter is configured to manually specify the default build system, and the subsequent build process will not prompt the user for selection.

Fastly cross compile

As we all know, although many projects maintained by autotools support cross-compilation, the configuration process of cross-compilation is very complicated. There are still many differences in different toolchain processing methods, and many pits will be stepped in the middle.

Even if you run through a toolchain’s cross-compilation, if you switch to another toolchain environment, it may take a long time, and if you use xmake, you usually only need two simple commands:

!> At present autotools supports cross-compilation of xmake, and other build systems such as cmake will be added later.

Cross compile android platform

$ xmake f -p android --trybuild=autotools [--ndk=xxx]
$ xmake

!> Among them, the –ndk parameter configuration is optional. If the user sets the ANDROID_NDK_HOME environment variable, or if the ndk is placed in ~/Library/Android/sdk/ndk-bundle, xmake can automatically detect it.

Is not it simple? If you think this is not much, then you can directly operate ./configure to configure cross-compilation. You can see this document for comparison: [Using NDK with other compilation systems] (https://developer.android .com/ndk/guides/other_build_systems # autoconf)

To put it bluntly, you probably have to do this, you may not be able to do it once:

$ export TOOLCHAIN=$NDK/toolchains/llvm/prebuilt/$HOST_TAG
$ export AR=$TOOLCHAIN/bin/aarch64-linux-android-ar
$ export AS=$TOOLCHAIN/bin/aarch64-linux-android-as
$ export CC=$TOOLCHAIN/bin/aarch64-linux-android21-clang
$ export CXX=$TOOLCHAIN/bin/aarch64-linux-android21-clang++
$ export LD=$TOOLCHAIN/bin/aarch64-linux-android-ld
$ export RANLIB=$TOOLCHAIN/bin/aarch64-linux-android-ranlib
$ export STRIP=$TOOLCHAIN/bin/aarch64-linux-android-strip
$ ./configure --host aarch64-linux-android
$ make

Cross compile iphoneos platform

$ xmake f -p iphoneos --trybuild=autotools
$ xmake

Cross-compile mingw platform

$ xmake f -p mingw --trybuild=autotools [--mingw=xxx]
$ xmake

Using other cross-compilation toolchains

$ xmake f -p cross --trybuild=autotools --sdk=/xxxx
$ xmake

For more cross compilation configuration details, please refer to the document: Cross Compilation, except for an additional --trybuild= parameter, all other cross-compilation configuration parameters are completely universal.

Passing user configuration parameters

We can use --tryconfigs= to pass additional configuration parameters of the user to the corresponding third-party build system. For example: autotools will be passed to . / Configure, cmake will be passed to the cmake command.

$ xmake f --trybuild=autotools --tryconfigs="-enable-shared=no"
$ xmake

For example, the above command, pass --enable-shared=no to./configure to disable dynamic library compilation.

In addition, for --cflags, --includedirs and --ldflags, you don’t need to pass --tryconfigs, you can pass the built-in parameters like xmake config --cflags= to pass through.

Examples of compiling other build system

General Compilation

In most cases, the compilation method after each docking system is consistent, except for the --trybuild= configuration parameter.

$ xmake f --trybuild=[autotools|cmake|meson|ninja|bazel|make|msbuild|xcodebuild]
$ xmake

!> We also need to make sure that the build tool specified by –trybuild is installed and working properly.

Building Android jni programs

If jni/Android.mk exists in the current project, then xmake can directly call ndk-build to build the jni library.

$ xmake f -p android --trybuild=ndkbuild [--ndk =]
$ xmake

*nix style command parameter input

The current input specification is referenced from: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html

Thank you very much @OpportunityLiu for your contribution. The current input method can support the following writing:

$ xmake -j8 -rvD

Before, I could only write:

$ xmake -j 8 -r -v -D

Tab command completion

In previous versions, only parameter names could be completed. You can now complete the parameter values and prompt the value list. For example, after typing the following command:

$ xmake f --plat = and

Press the tab key to complete the platform parameters and become

$ xmake f --plat = android

Force C code to be compiled as C ++

xmake adds a configuration parameter that specifies the type of source file and forces it to compile as a corresponding source file, such as compiling c code as c ++.

target("test")
    set_kind("binary")
    add_files("src/*.c", {sourcekind = "cxx"})

Changelog

New features

#675: Support to compile *.c as c++, add_files("*.c", {sourcekind = "cxx"}).
#681: Support compile xmake on msys/cygwin and add msys/cygwin platform
Add socket/pipe io modules and support to schedule socket/process/pipe in coroutine
#192: Try building project with the third-party buildsystem
Enable color diagnostics output for gcc/clang
#588: Improve project generator, xmake project -k ninja, support for build.ninja

Change

#665: Support to parse *nix style command options, thanks @OpportunityLiu
#673: Improve tab complete to support argument values
#680: Improve get.sh scripts and add download mirrors
Improve process scheduler
#651: Improve os/io module syserrors tips

Bugs fixed

Fix incremental compilation for checking the dependent file
Fix log output for parsing xmake-vscode/problem info
#684: Fix linker errors for android ndk on windows