TBOOX Open Source Project

• Xmake v2.7.3 Released, Package Components and C++ Modules Incremental Build Support

  2022-11-08

  xmake

  xmake  lua  C/C++  package  components

  Xmake is a lightweight cross-platform build utility based on Lua.

  It is very lightweight and has no dependencies because it has a built-in Lua runtime.

  It uses xmake.lua to maintain project builds and its configuration syntax is very simple and readable.

  We can use it to build project directly like Make/Ninja, or generate project files like CMake/Meson, and it also has a built-in package management system to help users solve the integrated use of C/C++ dependent libraries.

  Xmake = Build backend + Project Generator + Package Manager + [Remote|Distributed] Build + Cache
  

  Although not very precise, we can still understand Xmake in the following way:

  Xmake ~= Make/Ninja + CMake/Meson + Vcpkg/Conan + distcc + ccache/sccache
  

  • Github
  • Document

  

  Introduction of new features

  Package component support

  Introduction

  This new feature is intended to enable the integration of specific sub-libraries from a C/C++ package, and is generally used for library component integration in larger packages.

  This is because such packages provide a number of sub-libraries, not all of which are required by the user, and linking them all may be problematic.

  Although, previous versions were able to support the feature of sublibrary selection, e.g.

  add_requires("sfml~foo", {configs = {graphics = true, window = true}})
  add_requires("sfml~bar", {configs = {network = true}})
  
  target("foo")
      set_kind("binary")
      add_packages("sfml~foo")
  
  target("bar")
      set_kind("binary")
      add_packages("sfml~bar")
  

  This is done by custom configuration of each package, but there are some problems with this approach.

  1. sfml~foo and sfml~bar will be installed repeatedly as two separate packages, taking up double the disk space
  2. some common code will be compiled repeatedly, which will affect the efficiency of the installation
  3. if a target depends on both sfml~foo and sfml~bar, there will be link conflicts

  The impact of double-compilation and disk usage can be very high for very large package integrations such as boost, and can even lead to more than N times the disk usage if there are a large number of sub-library combinations.

  To solve this problem, Xmake has added a package component mode, which offers some of the following benefits.

  1. fast integration of any number of components in just one compile, greatly improving installation efficiency and reducing disk footprint
  2. component abstraction, across compilers and platforms, so users don’t need to worry about configuring link order dependencies between each sub library
  3. easier to use

  For more background details see: #2636

  Use package components

  For the user, using package components is very convenient because the user is not required to maintain the package, as long as the package is used, it is configured with the relevant set of components and we can quickly integrate and use it, e.g.

  add_requires("sfml")
  
  target("foo")
      set_kind("binary")
      add_packages("sfml", {components = "graphics"})
  
  target("bar")
      set_kind("binary")
      add_packages("sfml", {components = "network"})
  

  阅读中文 Read All

  ---

• Xmake v2.7.2 released, build third-party libraries more intelligently

  2022-10-09

  xmake

  xmake  lua  C/C++  trybuild  rule  cmake  autoconf

  Xmake is a lightweight cross-platform build utility based on Lua.

  It is very lightweight and has no dependencies because it has a built-in Lua runtime.

  It uses xmake.lua to maintain project builds and its configuration syntax is very simple and readable.

  We can use it to build project directly like Make/Ninja, or generate project files like CMake/Meson, and it also has a built-in package management system to help users solve the integrated use of C/C++ dependent libraries.

  Xmake = Build backend + Project Generator + Package Manager + [Remote|Distributed] Build + Cache
  

  Although not very precise, we can still understand Xmake in the following way:

  Xmake ~= Make/Ninja + CMake/Meson + Vcpkg/Conan + distcc + ccache/sccache
  

  • Github
  • Document

  Introduction of new features

  Building third party libraries more intelligently

  In previous versions, Xmake provided a TryBuild mode that allowed you to use Xmake to try to build third-party projects maintained by autoconf/cmake/meson etc. directly without xmake.lua.

  In effect, this means that Xmake detects the corresponding build system and invokes commands such as cmake to do so, but it will help the user to simplify the configuration operation, plus it will interface with xmake’s cross-compilation toolchain configuration.

  However, this mode has a certain failure rate, which can lead to build failure if, for example

  1. the project code itself is flawed, resulting in a compilation error
  2. the project code does not support the current platform
  3. the build script is flawed
  4. specific configuration parameters are missing
  5. a missing dependency library that needs to be installed by the user
  6. the compiler version is too low and does not support some of the code

  The TryBuild mode usually handles these cases, but in this new version we have introduced a new mechanism to the TryBuild mode to improve the build logic by reusing build scripts from the xmake-repo repository.

  It roughly handles the process in the following way.

  1. execute the xmake command in the third-party source repository directory
  2. xmake gets the directory name and tries to resolve the project name and version
  3. try to match an existing package from the xmake-repo repository
  4. If the match is successful, build directly using the build logic in the package
  5. if no match is made, fall back to the original TryBuild logic

  What is the benefit of this, if the match is successful, we can solve all the problems mentioned above.

  Even if the current project source code does not support a given platform, or if the source code and build script are flawed in some way, Xmake will automatically patch in a specific patch to fix it and bring in the required dependencies to ensure that it will definitely compile in one click.

  Let’s take a look at the libjpeg library as an example.

  The first step is to download the corresponding source code package

  $ wget https://jaist.dl.sourceforge.net/project/libjpeg-turbo/2.1.4/libjpeg-turbo-2.1.4.tar.gz
  $ tar -xvf libjpeg-turbo-2.1.4.tar.gz
  $ cd libjpeg-turbo-2.1.4
  

  Enter the directory and execute the Xmake command

  Xmake will prompt the user if it detects that it is the libjpeg library, and whether to build it as libjpeg 2.1.4.

  ruki-2:libjpeg-turbo-2.1.4 ruki$ xmake
  note: libjpeg-turbo 2.1.4 in xmake-repo found, try building it or you can run ``xmake f --trybuild=` to set buildsystem (pass -y or --confirm=y/n/d to skip confirm)?
  please input: y (y/n)
  

  We hit enter to confirm to continue the build.

  阅读中文 Read All

  ---

• Xmake v2.7.1 Released, Better C++ Modules Support

  2022-08-25

  xmake

  xmake  lua  C/C++  remote  ccache  C++20  Modules  headerunits  fs-watcher

  Xmake is a lightweight cross-platform build utility based on Lua.

  It is very lightweight and has no dependencies because it has a built-in Lua runtime.

  It uses xmake.lua to maintain project builds and its configuration syntax is very simple and readable.

  We can use it to build project directly like Make/Ninja, or generate project files like CMake/Meson, and it also has a built-in package management system to help users solve the integrated use of C/C++ dependent libraries.

  Xmake = Build backend + Project Generator + Package Manager + [Remote|Distributed] Build + Cache
  

  Although not very precise, we can still understand Xmake in the following way:

  Xmake ~= Make/Ninja + CMake/Meson + Vcpkg/Conan + distcc + ccache/sccache
  

  • Github
  • Document

  Introduction of new features

  In this release, we have refactored and improved the C++20 Modules implementation, improved the dependency graph parsing of module files, added support for STL and User HeaderUnits, and made the CMakelists/compile_commands generator support C++ Modules.

  In addition, we’ve added an xmake watch plugin that can monitor current project file updates in real time, automatically trigger incremental builds, or run some custom commands.

  

  阅读中文 Read All

  ---

• Xmake v2.6.6 Released, Support Distributed Compilation and Build Cache

  2022-05-24

  xmake

  xmake  lua  C/C++  remote  ccache  distributed-compilation

  Xmake is a lightweight cross-platform build utility based on Lua.

  It is very lightweight and has no dependencies because it has a built-in Lua runtime.

  It uses xmake.lua to maintain project builds and its configuration syntax is very simple and readable.

  We can use it to build project directly like Make/Ninja, or generate project files like CMake/Meson, and it also has a built-in package management system to help users solve the integrated use of C/C++ dependent libraries.

  Xmake = Build backend + Project Generator + Package Manager + [Remote|Distributed] Build + Cache
  

  Although not very precise, we can still understand Xmake in the following way:

  Xmake ~= Make/Ninja + CMake/Meson + Vcpkg/Conan + distcc + ccache/sccache
  

  • Github
  • Document

  Introduction of new features

  In this version, we have added a lot of heavyweight new features:

  • Distributed compilation
  • Local compilation cache
  • Remote compilation cache

  With these features, we can compile large C/C++ projects faster.

  In addition, they are completely cross-platform, support not only gcc/clang but also msvc, and there is no any third-party dependency except the compiler, which is very convenient to use.

  Therefore, using Xmake is equivalent to using distcc/ccache/sccache at the same time.

  Compared with these third-party tools, Xmake fully supports Windows and msvc, which eliminates platform differences, independent process calls, and the overhead of additional daemon processes.

  In addition to these features, the new version of Xmake also adds support for compiling Keil/C51 projects, as well as support for the nvc/nvc++/nvfortran compilers in the nvidia-hpc-sdk toolchain.

  Support user authentication for remote compilation

  In the last version, we initially supported remote compilation, but did not provide user authentication support, which would bring some security issues. Therefore, in this version, we added user authentication support.

  At present, Xmake mainly provides the following authentication mechanisms. In addition, it is also effective for distributed compilation and remote caching.

  1. Token authentication
  2. Password authentication
  3. Trusted host verification

  Token authentication

  This is also the default recommended method, which is more secure, more convenient to configure and connect, and does not need to enter a password every time you connect.

  When we execute the xmake service command, a server and client configuration file will be generated by default, and a default token will be automatically generated, so the local direct connection does not require any configuration.

  阅读中文 Read All

  ---

• Xmake v2.6.5 released, Support remote compilation

  2022-04-24

  xmake

  xmake  lua  C/C++  cargo  rust  remote-compilation

  Xmake is a lightweight cross-platform build utility based on Lua.

  It is very lightweight and has no dependencies because it has a built-in Lua runtime.

  It uses xmake.lua to maintain project builds and its configuration syntax is very simple and readable.

  We can use it to build project directly like Make/Ninja, or generate project files like CMake/Meson, and it also has a built-in package management system to help users solve the integrated use of C/C++ dependent libraries.

  Xmake = Build backend + Project Generator + Package Manager
  

  • Github
  • Document

  Introduction of new features

  Remote Compilation Support

  The new version provides remote compilation support, which allows us to compile code on a remote server, run and debug remotely.

  The server can be deployed on Linux/MacOS/Windows to enable cross-platform compilation, e.g. compile and run Windows programs on Linux and macOS/Linux programs on Windows.

  It is more stable and smoother to use than ssh remote login compilation, no lagging of ssh terminal input due to network instability, and allows for quick local editing of code files.

  We can even seamlessly implement remote compilation in editors and IDEs such as vs/sublime/vscode/idea without relying on the IDE’s own support for remote compilation.

  Start service

  $ xmake service
  <remote_build_server>: listening 0.0.0.0:9096 ...
  

  We can also turn on the service while displaying back detailed log messages.

  $ xmake service -vD
  <remote_build_server>: listening 0.0.0.0:9096 ...
  

  Start the service in Daemon mode

  $ xmake service --start
  $ xmake service --restart
  $ xmake service --stop
  

  Configure the server

  We start by running the xmake service command, which automatically generates a default service.conf configuration file, stored in ~/.xmake/service.conf.

  Then, we edit it to fix the server’s listening port (optional).

  {
      logfile = "/Users/ruki/.xmake/service/logs.txt",
      remote_build = {
          server = {
              listen = "0.0.0.0:9096"
          }
      }
  }
  

  Configure the client

  We still edit this file ~/.xmake/service.conf to configure the address of the server to which the client needs to connect.

  {
      logfile = "/Users/ruki/.xmake/service/logs.txt",
      remote_build = {
          client = {
              connect = "192.168.56.101:9096",
          }
      }
  }
  

  Import the given configuration file

  We can also import the given configuration file by using the following command.

  $ xmake service --config=/tmp/service.conf
  

  Connect to the remote server

  Next, we just need to go into the root directory of the project we need to compile remotely and execute the xmake service --connect command to make the connection.

  $ xmake create test
  $ cd test
  $ xmake service --connect 
  <remote_build_client>: connect 192.168.56.110:9096 ...
  <remote_build_client>: connected!
  <remote_build_client>: sync files in 192.168.56.110:9096 ...
  Scanning files ...
  Comparing 3 files ...
      [+]: src/main.cpp
      [+]: .gitignore
      [+]: xmake.lua
  3 files has been changed!
  Archiving files .
  Uploading files with 1372 bytes ...
  <remote_build_client>: sync files ok!
  

  Remote build project

  Once the connection is successful, we can build remotely as if we were building locally as normal.

  ```console $ xmake

  : run xmake in 192.168.56.110:9096 ... checking for platform ... macosx checking for architecture ... ... x86_64 ... checking for Xcode directory ... /Applications/Xcode.app checking for Codesign Identity of Xcode ... Apple Development: waruqi@gmail.com (T3NA4MRVPU) ... checking for SDK version of Xcode for macosx (x86_64) ... 11.3 ... checking for Minimal target version of Xcode for macosx (x86_64) ... 11.4 [ 25%]: ccache compiling.release src/main.cpp [ 50%]: linking.release test [ 100%]: build ok! : run command ok! ```'' [ 25% ]: ccache compiling.release src/main.cpp [ 50%]: linking.release test [100%]: build ok!
  阅读中文 Read All

  ---