众所周知,干我们这行的在项目前期进行驱动移植时,往往使用编译模块的方法可以节省大量的时间。我们可以单独编译ko甚至利用Android的构建系统的脚本编译出ko,然后手动insmod安装到手机里。然而当我们full build image的时候,会发现有时候ko并没有merge到image中,要么就发现这个ko没有在开机时自动insmod!

针对这个问题,本文将围绕这个问题展开

ko编译原理

我们首先要明确,ko是如何被编译出来的。如果内核外的ko用Android.bp或者Android.mk,那直接mm就可以编译出来。

树外驱动的编译原理

高通平台

我们可以在树外驱动的Android.mk中得到大概如下的代码

DISPLAY_SELECT := CONFIG_DRM_MSM=m

LOCAL_PATH := $(call my-dir)
LOCAL_MODULE_DDK_BUILD := true
include $(CLEAR_VARS)

# This makefile is only for DLKM
ifneq ($(findstring vendor,$(LOCAL_PATH)),)

ifneq ($(findstring opensource,$(LOCAL_PATH)),)
	DISPLAY_BLD_DIR := $(TOP)/vendor/qcom/opensource/display-drivers
endif # opensource

DLKM_DIR := $(TOP)/device/qcom/common/dlkm

LOCAL_ADDITIONAL_DEPENDENCIES := $(wildcard $(LOCAL_PATH)/**/*) $(wildcard $(LOCAL_PATH)/*)

# Build display.ko as msm_drm.ko
###########################################################
# This is set once per LOCAL_PATH, not per (kernel) module
KBUILD_OPTIONS := DISPLAY_ROOT=$(DISPLAY_BLD_DIR)
KBUILD_OPTIONS += MODNAME=msm_drm
KBUILD_OPTIONS += BOARD_PLATFORM=$(TARGET_BOARD_PLATFORM)
KBUILD_OPTIONS += $(DISPLAY_SELECT)

###########################################################
include $(CLEAR_VARS)
LOCAL_SRC_FILES   := $(wildcard $(LOCAL_PATH)/**/*) $(wildcard $(LOCAL_PATH)/*)
LOCAL_MODULE              := msm_drm.ko
LOCAL_MODULE_KBUILD_NAME  := msm_drm.ko
LOCAL_MODULE_TAGS         := optional
LOCAL_MODULE_DEBUG_ENABLE := true
LOCAL_MODULE_PATH         := $(KERNEL_MODULES_OUT)

include $(DLKM_DIR)/Build_external_kernelmodule.mk
###########################################################
endif # DLKM check

在这个Android.mk中除了定义了基本的LOCAL_*的参数外,还有一句

include $(DLKM_DIR)/Build_external_kernelmodule.mk

在高通平台中的树外驱动,依赖于一个ko的编译模板,也就Build_externel_kernelmodule.mk,它位device/qcom/common/dlkm/

而其中最核心的就是下面这个指令

./build/build_module.sh $(kbuild_options)

我们先看看usage

# Usage:
#   build/build_module.sh <make options>*
# or:
#   OUT_DIR=<out dir> DIST_DIR=<dist dir> build/build_module.sh <make options>*
#
# Example:
#   OUT_DIR=output DIST_DIR=dist build/build_module.sh -j24 V=1#
#
# The following environment variables are considered during execution:
#
#   BUILD_CONFIG
#     Build config file to initialize the build environment from. The location
#     is to be defined relative to the repo root directory.
#     Defaults to 'build.config'.
#
#   MODULE_CONFIG
#     Build config file for module to initialize the build environment from.
#     The location is to be defined relative to the repo root directory.
#     Defaults to 'module.config'.
#
#   EXT_MODULES
#     Space separated list of external kernel modules to be build.
#
#   UNSTRIPPED_MODULES
#     Space separated list of modules to be copied to <DIST_DIR>/unstripped
#     for debugging purposes.
#
#   INPLACE_COMPILE
#     Conditional flag for in-place compilation. When set to 'y', the intermediate
#     files and final module output will be stored in the module source directory.
#     In-place compilation is tot supported in DDK build.
#
#   MODULE_OUT
#     Location to place compiled module output. When this option is specified,
#     Only one EXT_MODULES may be specified. A symlink is created from the
#     output Kbuild will use to MODULE_OUT.
#
#   OUT_DIR
#     Location to store the intermediate kernel environemnt for module compilation.
#
# Environment variables to influence the stages of the kernel build.
#
#   SKIP_MRPROPER
#     if defined, skip `make mrproper`
#
#   INSTALL_MODULE_HEADERS
#     if defined, install uapi headers from the module.
#
#   BUILD_DTBS
#     if defined, install uapi headers from the module.

而编译模块的底层逻辑,则区分不同的kernel版本有所叙别,但是暴露给Android上层的Build_external_kernelmodule.mk Android.mk 都是固定的。而不一样的 地方如下

kernel-6.1 之前,使用make

    make -C ${EXT_MOD} M=${EXT_MOD_REL} KERNEL_SRC=${ROOT_DIR}/${KERNEL_DIR}  \
                      O=${OUT_DIR} "${TOOL_ARGS[@]}"                         \
                      INSTALL_HDR_PATH="${KERNEL_UAPI_HEADERS_DIR}/usr"      \
                      ${MAKE_ARGS} headers_install

而kernel-6.1之后,使用bazel

   # Run the dist command passing in the output directory from Android build system
    ./tools/bazel run "${build_flags[@]}" "$build_target" \
      -- --dist_dir="${OUT_DIR}/${EXT_MOD_REL}"

MTK平台

与高通平台类似,在MTK平台也是存在一个ko编译模板

include $(CLEAR_VARS)

LOCAL_MODULE := $(WIFI_NAME).ko
LOCAL_PROPRIETARY_MODULE := true
LOCAL_MODULE_OWNER := mtk

LOCAL_REQUIRED_MODULES := $(WIFI_CHRDEV_MODULE)

ifeq ($(CONNAC_VER), 3_0)
	LOCAL_REQUIRED_MODULES += wlan_page_pool.ko
	LOCAL_REQUIRED_MODULES += conninfra.ko
	LOCAL_REQUIRED_MODULES += connfem.ko
else ifeq ($(CONNAC_VER), 2_0)
	LOCAL_REQUIRED_MODULES += conninfra.ko
	LOCAL_REQUIRED_MODULES += connfem.ko
else
	LOCAL_REQUIRED_MODULES += wmt_drv.ko
endif

include $(MTK_KERNEL_MODULE)

MTK_KERNEL_MODULE 也就是

MTK_KERNEL_MODULE := $(LOCAL_PATH)/build_ko.mk

核心在于

cd kernel && OUT_DIR=$(KOUT) BUILD_CONFIG=$(PRIVATE_KERNEL_BUILD_CONFIG) ./../device/mediatek/build/build/tools/build_kernel.sh $(OPTS) src=$(PRIVATE_LOCAL_PATH) && cd ..

而build_kernel.sh中我们可以看到

(cd ${OUT_DIR} && make O=${OUT_DIR} "${TOOL_ARGS[@]}" "${MAKE_ARGS[@]}")

目前我所接触到的一些MTK项目,6.1、5.15、6.6项目树外驱动仍然使用的make进行编译!有可能未来会变更

树内驱动的编译原理

关于树内驱动的编译,这里只说明一下高通平台的

RECOMPILE_KERNEL=1 LTO=thin \
./kernel_platform/build/android/prepare_vendor.sh \
bengal consolidate

PS: consolidate就类似于userdebug,而user版本有些基线使用的是gki,有的使用的是perf

if [ "${RECOMPILE_KERNEL}" == "1" ]; then
  echo
  echo "  Recompiling kernel"

  PLATFORM_SECURITY_PATCH=$(get_build_var PLATFORM_SECURITY_PATCH)
  echo "  PLATFORM_SECURITY_PATCH: ${PLATFORM_SECURITY_PATCH}"
  echo ${PLATFORM_SECURITY_PATCH} > ${ANDROID_BUILD_TOP}/out/target/product/${DEVICE_NAME}/platform_security_patch.txt

  # shellcheck disable=SC2086
  "${ROOT_DIR}/build_with_bazel.py" \
    -t "$KERNEL_TARGET" "$KERNEL_VARIANT" $LTO_KBUILD_ARG $EXTRA_KBUILD_ARGS --define=FACTORY_BUILD=${FACTORY_BUILD} \
    --define=PLATFORM_SECURITY_PATCH=${PLATFORM_SECURITY_PATCH} \
    --out_dir "${ANDROID_KP_OUT_DIR}"

  COPY_NEEDED=1
fi

这里以6.1内核继续展开,6.1之前的的使用make编译的,可以自行了解

核心在:

    def run_targets(self, targets):
        """Run "bazel run" on all targets in serial (since bazel run cannot have multiple targets)"""
        for target in targets:
            # Set the output directory based on if it's a host target
            if any(
                re.match(r"//{}:.*_{}_dist".format(self.kernel_dir, h), target.bazel_label)
                    for h in HOST_TARGETS
            ):
                out_dir = target.get_out_dir("host")
            else:
                out_dir = target.get_out_dir("dist")
            self.bazel(
                "run",
                [target],
                extra_options=self.user_opts,
                bazel_target_opts=["--dist_dir", out_dir]
            )
            self.write_opts(out_dir)

我们需要理解这段的逻辑,因为只有了解这段逻辑,我们才能有一些客制化的需求才能实现!比如向内核编译时传参数

ko如何打包到image中

随着google GKI的实行,第三方的驱动必须以ko的形式实现。故我们需要了解

  1. 这些ko打包到哪个image中?

  2. 这些ko是如何打包进image的?

第一点,ko会打包到vendor_dlkm.img中,然后因为是动态分区的原因,最终会merge到 super.img中

function build_vendor_dlkm() {
  local vendor_dlkm_archive=$1

  echo "========================================================"
  echo " Creating vendor_dlkm image"

  create_modules_staging "${VENDOR_DLKM_MODULES_LIST}" "${MODULES_STAGING_DIR}" \
    "${VENDOR_DLKM_STAGING_DIR}" "${VENDOR_DLKM_MODULES_BLOCKLIST}"

  local vendor_dlkm_modules_root_dir=$(echo ${VENDOR_DLKM_STAGING_DIR}/lib/modules/*)
  local vendor_dlkm_modules_load=${vendor_dlkm_modules_root_dir}/modules.load
  if [ -f ${vendor_dlkm_modules_root_dir}/modules.blocklist ]; then
    cp ${vendor_dlkm_modules_root_dir}/modules.blocklist ${DIST_DIR}/vendor_dlkm.modules.blocklist
  fi

  # Modules loaded in vendor_boot (and optionally system_dlkm if dedup_dlkm_modules)
  # should not be loaded in vendor_dlkm.
  if [ -f ${DIST_DIR}/modules.load ]; then
    local stripped_modules_load="$(mktemp)"
    ! grep -x -v -F -f ${DIST_DIR}/modules.load \
      ${vendor_dlkm_modules_load} > ${stripped_modules_load}
    mv -f ${stripped_modules_load} ${vendor_dlkm_modules_load}
  fi

  cp ${vendor_dlkm_modules_load} ${DIST_DIR}/vendor_dlkm.modules.load

  if [ -e ${vendor_dlkm_modules_root_dir}/modules.blocklist ]; then
    cp ${vendor_dlkm_modules_root_dir}/modules.blocklist \
      ${DIST_DIR}/vendor_dlkm.modules.blocklist
  fi

  local vendor_dlkm_props_file

  local vendor_dlkm_default_fs_type="ext4"
  if [[ "${VENDOR_DLKM_FS_TYPE}" != "ext4" && "${VENDOR_DLKM_FS_TYPE}" != "erofs" ]]; then
    echo "WARNING: Invalid VENDOR_DLKM_FS_TYPE = ${VENDOR_DLKM_FS_TYPE}"
    VENDOR_DLKM_FS_TYPE="${vendor_dlkm_default_fs_type}"
    echo "INFO: Defaulting VENDOR_DLKM_FS_TYPE to ${VENDOR_DLKM_FS_TYPE}"
  fi

  if [ -z "${VENDOR_DLKM_PROPS}" ]; then
    vendor_dlkm_props_file="$(mktemp)"
    echo -e "vendor_dlkm_fs_type=${VENDOR_DLKM_FS_TYPE}\n" >> ${vendor_dlkm_props_file}
    echo -e "use_dynamic_partition_size=true\n" >> ${vendor_dlkm_props_file}
    if [[ "${VENDOR_DLKM_FS_TYPE}" == "ext4" ]]; then
      echo -e "ext_mkuserimg=mkuserimg_mke2fs\n" >> ${vendor_dlkm_props_file}
      echo -e "ext4_share_dup_blocks=true\n" >> ${vendor_dlkm_props_file}
    fi
  else
    vendor_dlkm_props_file="${VENDOR_DLKM_PROPS}"
    if [[ -f "${ROOT_DIR}/${vendor_dlkm_props_file}" ]]; then
      vendor_dlkm_props_file="${ROOT_DIR}/${vendor_dlkm_props_file}"
    elif [[ "${vendor_dlkm_props_file}" != /* ]]; then
      echo "VENDOR_DLKM_PROPS must be an absolute path or relative to ${ROOT_DIR}: ${vendor_dlkm_props_file}"
      exit 1
    elif [[ ! -f "${vendor_dlkm_props_file}" ]]; then
      echo "Failed to find VENDOR_DLKM_PROPS: ${vendor_dlkm_props_file}"
      exit 1
    fi
  fi

  # Copy etc files to ${DIST_DIR} and ${VENDOR_DLKM_STAGING_DIR}/etc
  if [[ -n "${VENDOR_DLKM_ETC_FILES}" ]]; then
    local etc_files_dst_folder="${VENDOR_DLKM_STAGING_DIR}/etc"
    mkdir -p "${etc_files_dst_folder}"
    cp ${VENDOR_DLKM_ETC_FILES} "${etc_files_dst_folder}"
    cp ${VENDOR_DLKM_ETC_FILES} "${DIST_DIR}"
  fi

  build_image "${VENDOR_DLKM_STAGING_DIR}" "${vendor_dlkm_props_file}" \
    "${DIST_DIR}/vendor_dlkm.img" /dev/null

  avbtool add_hashtree_footer \
    --partition_name vendor_dlkm \
    --hash_algorithm sha256 \
    --image "${DIST_DIR}/vendor_dlkm.img"

  if [ -n "${vendor_dlkm_archive}" ]; then
    # Archive vendor_dlkm_staging_dir
    tar -czf "${DIST_DIR}/vendor_dlkm_staging_archive.tar.gz" -C "${VENDOR_DLKM_STAGING_DIR}" .
  fi
}

动态加载的ko是cp到vendor_dlkm分区里面。所以,我们可以执行make vendor_dlkmimage的方式,先编译出来vendor_dlkm.img,并且这个image是merge到super分区了。

当我们编译出vendor_dlkm.img后,也可以通过指令单刷这个镜像!


adb reboot fastboot //进入fastbootd mode
fastboot flash vendor_dlkm vendor_dlkm.img

注意:由于google的android版本变化以及kernel版本变化,编译方式以及编译的产物路径都会存在一定的差异,本文仅供大家参考!