Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[datalog] Move DataLogReaderThread from glass to new libdatalog #7641

Open
wants to merge 66 commits into
base: 2027
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 65 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
43f5103
move datalogreaderthread to wpiutil
DeltaDizzy Jan 5, 2025
39be954
fix datalog rader references in sysid
DeltaDizzy Jan 5, 2025
910d24f
run wpiformat
DeltaDizzy Jan 5, 2025
638557f
point datalogtool at wpiutil
DeltaDizzy Jan 5, 2025
9bbe719
run wpiformat on datalogtool
DeltaDizzy Jan 5, 2025
f21d596
move DataLog classes to wpi::log namespace
DeltaDizzy Jan 7, 2025
c6974c5
wpiformat
DeltaDizzy Jan 7, 2025
85a9edf
create libdatalog, move datalog from wpiutil
DeltaDizzy Jan 8, 2025
a7f51b0
1st try at bazel
DeltaDizzy Jan 9, 2025
3d56e72
include deps and sources for bazel
DeltaDizzy Jan 9, 2025
2928e0d
move datalog examples to libdatalog
DeltaDizzy Jan 9, 2025
6835116
add datalog examples and jni
DeltaDizzy Jan 9, 2025
ac84785
move java datalog lib
DeltaDizzy Jan 9, 2025
0c649a0
mvoe filelogger (cpp) and cpp jni
DeltaDizzy Jan 9, 2025
c9309bf
add build.gradle
DeltaDizzy Jan 9, 2025
a101ee5
move tests
DeltaDizzy Jan 9, 2025
374ebf9
fix tests cmake
DeltaDizzy Jan 9, 2025
93935d2
Add gradle build
spacey-sooty Jan 9, 2025
f0c58b4
update java datalog package
DeltaDizzy Jan 9, 2025
a31527b
import wpiutil jni
DeltaDizzy Jan 9, 2025
a6ecef9
add datalog to ntcore gradle
DeltaDizzy Jan 9, 2025
29ef6cc
gradle updates
DeltaDizzy Jan 9, 2025
6fec7b9
revert to og gradle
DeltaDizzy Jan 9, 2025
3fd78f4
jni hell part 2
DeltaDizzy Jan 9, 2025
2de89e4
almost there!
DeltaDizzy Jan 9, 2025
eef626d
libdatalog builds
DeltaDizzy Jan 9, 2025
4ed06d0
SUCCESS
DeltaDizzy Jan 9, 2025
9cd5ca4
add to new commands
DeltaDizzy Jan 9, 2025
63f5af6
newline at end of build.bazel
DeltaDizzy Jan 9, 2025
72228c8
Revert "move kInvalidFile declaration to header"
DeltaDizzy Jan 9, 2025
f34418c
uni time
DeltaDizzy Jan 12, 2025
32b8f08
constexpr invalid file
DeltaDizzy Jan 17, 2025
7f210f6
use macro in fs.h
DeltaDizzy Jan 17, 2025
f02ddbe
Add more DataLogJNI implementations
DeltaDizzy Jan 25, 2025
e424105
move jni load and add ntcore dependency
DeltaDizzy Jan 26, 2025
5f8f548
fix jni path
DeltaDizzy Jan 26, 2025
14867d4
fix java jni and bazel
DeltaDizzy Jan 26, 2025
a4ce616
re-add wpiutiljni to datalogjni
DeltaDizzy Jan 26, 2025
a3f4c99
wpiformat
DeltaDizzy Jan 26, 2025
8ee9c3f
wpiutil format
DeltaDizzy Jan 26, 2025
578a962
make wpiutil link last
DeltaDizzy Jan 26, 2025
18f1da8
reorder dlt deps
DeltaDizzy Jan 26, 2025
628537f
sysid deps
DeltaDizzy Jan 26, 2025
d7ac8a2
move filelogger from wpiutil to datalog (java)
DeltaDizzy Jan 26, 2025
24ff046
fix jni names
DeltaDizzy Jan 26, 2025
0679055
Update libdatalog/BUILD.bazel
DeltaDizzy Jan 26, 2025
14d38dc
dont extend wpiutiljni
DeltaDizzy Jan 26, 2025
add82d9
gradle tests fix
DeltaDizzy Jan 26, 2025
fa553b8
add datalog to sysid tests
DeltaDizzy Jan 27, 2025
4c93e61
fix bazel build for libdatalog, epilogue, ntcore, wpiutil
DeltaDizzy Jan 27, 2025
cf15228
rename to datalog
DeltaDizzy Jan 27, 2025
83f56e3
format datalog cmake
DeltaDizzy Jan 27, 2025
464d557
format bazels
DeltaDizzy Jan 27, 2025
5f10384
add datalog to wpilibj
DeltaDizzy Jan 27, 2025
17127e9
fix java rules def
DeltaDizzy Jan 27, 2025
9ea037e
format bazels
DeltaDizzy Jan 28, 2025
518cdde
add bazel tests
DeltaDizzy Jan 28, 2025
7256833
format wpilibc bazel
DeltaDizzy Jan 28, 2025
22aaec0
move datalog headers to wpi/datalog
DeltaDizzy Jan 28, 2025
8556f43
file file logger test includes
DeltaDizzy Jan 31, 2025
12a776d
fix jni includes
DeltaDizzy Jan 31, 2025
b816c21
filelogger namespaces
DeltaDizzy Jan 31, 2025
700d148
run wpiformat
DeltaDizzy Feb 1, 2025
497f403
last include
DeltaDizzy Feb 1, 2025
132cb3f
remove old usage reporting imports
DeltaDizzy Feb 9, 2025
8f10a64
move datalog spec
DeltaDizzy Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,8 @@ set(WPIUNITS_DEP_REPLACE_IMPL "find_dependency(wpiunits)")
set(WPIUTIL_DEP_REPLACE "find_dependency(wpiutil)")
add_subdirectory(wpiutil)

add_subdirectory(datalog)

if(WITH_NTCORE)
set(NTCORE_DEP_REPLACE "find_dependency(ntcore)")
set(WPINET_DEP_REPLACE "find_dependency(wpinet)")
Expand Down
106 changes: 106 additions & 0 deletions datalog/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
load("@rules_cc//cc:defs.bzl", "cc_binary", "cc_library", "cc_test")
load("@rules_java//java:defs.bzl", "java_binary")
load("@rules_python//python:defs.bzl", "py_binary")
load("//shared/bazel/rules:java_rules.bzl", "wpilib_java_junit5_test")
load("//shared/bazel/rules:jni_rules.bzl", "wpilib_jni_cc_library", "wpilib_jni_java_library")

cc_library(
name = "datalog.static",
srcs = glob(
["src/main/native/cpp/**"],
exclude = ["src/main/native/cpp/jni/**"],
),
hdrs = glob(["src/main/native/include/**"]),
includes = [
"src/main/native/cpp",
"src/main/native/include",
],
strip_include_prefix = "src/main/native/include",
visibility = ["//visibility:public"],
deps = [
"//wpiutil:wpiutil.static",
],
)

wpilib_jni_cc_library(
name = "datalogjni",
srcs = glob(["src/main/native/cpp/jni/**"]),
java_dep = ":datalog-java",
visibility = ["//visibility:public"],
deps = [
":datalog.static",
],
)

wpilib_jni_java_library(
name = "datalog-java",
srcs = glob(["src/main/java/**/*.java"]),
native_libs = [":datalogjni"],
visibility = ["//visibility:public"],
deps = [
"//wpiutil:wpiutil-java",
"@maven//:us_hebi_quickbuf_quickbuf_runtime",
],
)

py_binary(
name = "datalog",
srcs = ["examples/printlog/datalog.py"],
tags = ["manual"],
)

cc_binary(
name = "printlog",
srcs = ["examples/printlog/printlog.cpp"],
deps = [
":datalog.static",
"//wpiutil:wpiutil.static",
],
)

cc_binary(
name = "writelog",
srcs = ["examples/writelog/writelog.cpp"],
deps = [
":datalog.static",
"//wpiutil:wpiutil.static",
],
)

java_binary(
name = "printlog-java",
srcs = ["src/printlog/java/printlog/PrintLog.java"],
main_class = "printlog.PrintLog",
deps = [
":datalog-java",
],
)

cc_test(
name = "datalog-cpp-test",
size = "small",
srcs = glob([
"src/test/native/**/*.cpp",
"src/test/native/**/*.h",
]),
tags = [
"exclusive",
"no-asan",
"no-tsan",
],
deps = [
":datalog.static",
"//thirdparty/googletest:googletest.static",
"//wpiutil:wpiutil-testlib",
],
)

wpilib_java_junit5_test(
name = "datalog-java-test",
srcs = glob(["src/test/java/**/*.java"]),
tags = ["exclusive"],
deps = [
":datalog-java",
"//wpiutil:wpiutil-java",
],
)
108 changes: 108 additions & 0 deletions datalog/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
project(datalog)

include(CompileWarnings)

file(GLOB datalog_native_src src/main/native/cpp/*.cpp)

file(GLOB datalog_jni_src src/main/native/cpp/jni/DataLogJNI.cpp)
list(REMOVE_ITEM datalog_native_src ${datalog_jni_src})

add_library(datalog ${datalog_native_src})
set_target_properties(datalog PROPERTIES DEBUG_POSTFIX "d")

target_compile_features(datalog PUBLIC cxx_std_20)
if(MSVC)
target_compile_options(
datalog
PUBLIC /permissive- /Zc:preprocessor /Zc:__cplusplus /Zc:throwingNew /MP /bigobj /utf-8
)
target_compile_definitions(datalog PRIVATE -D_CRT_SECURE_NO_WARNINGS)
endif()
wpilib_target_warnings(datalog)

target_include_directories(
datalog
PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/src/main/native/include>
)

target_link_libraries(datalog PRIVATE wpiutil)

subdir_list(datalog_examples "${CMAKE_CURRENT_SOURCE_DIR}/examples")
foreach(example ${datalog_examples})
file(GLOB datalog_example_src examples/${example}/*.cpp)
if(datalog_example_src)
add_executable(datalog_${example} ${datalog_example_src})
wpilib_target_warnings(datalog_${example})
target_link_libraries(datalog_${example} datalog wpiutil)
set_property(TARGET datalog_${example} PROPERTY FOLDER "examples")
endif()
endforeach()

# Java bindings
if(WITH_JAVA)
include(UseJava)

set(CMAKE_JNI_TARGET true)

file(GLOB_RECURSE JAVA_SOURCES src/main/java/*.java)
file(GLOB QUICKBUF_JAR ${WPILIB_BINARY_DIR}/wpiutil/thirdparty/quickbuf/*.jar)

add_jar(
datalog_jar
${JAVA_SOURCES}
# INCLUDE_JARS ${JACKSON_JARS} ${QUICKBUF_JAR}
INCLUDE_JARS wpiutil_jar ${QUICKBUF_JAR}
OUTPUT_NAME datalog
OUTPUT_DIR ${WPILIB_BINARY_DIR}/${java_lib_dest}
GENERATE_NATIVE_HEADERS datalog_jni_headers
)
set_property(TARGET datalog_jar PROPERTY FOLDER "java")

install_jar(datalog_jar DESTINATION ${java_lib_dest})
install_jar_exports(TARGETS datalog_jar FILE datalog_jar.cmake DESTINATION share/datalog)

add_library(datalogjni ${datalog_jni_src})
wpilib_target_warnings(datalogjni)
target_link_libraries(datalogjni PUBLIC datalog wpiutil)

set_property(TARGET datalogjni PROPERTY FOLDER "libraries")

target_link_libraries(datalogjni PRIVATE datalog_jni_headers)
add_dependencies(datalogjni datalog_jar)

install(TARGETS datalogjni EXPORT datalogjni)
export(TARGETS datalogjni FILE datalogjni.cmake NAMESPACE datalogjni::)
endif()

if(WITH_JAVA_SOURCE)
include(UseJava)
include(CreateSourceJar)
add_source_jar(
datalog_src_jar
BASE_DIRECTORIES ${CMAKE_CURRENT_SOURCE_DIR}/src/main/java
OUTPUT_NAME datalog-sources
OUTPUT_DIR ${WPILIB_BINARY_DIR}/${java_lib_dest}
)
set_property(TARGET datalog_src_jar PROPERTY FOLDER "java")

install_jar(datalog_src_jar DESTINATION ${java_lib_dest})
endif()

install(TARGETS datalog EXPORT datalog)
export(TARGETS datalog FILE datalog.cmake NAMESPACE datalog::)

configure_file(datalog-config.cmake.in ${WPILIB_BINARY_DIR}/datalog-config.cmake)
install(FILES ${WPILIB_BINARY_DIR}/datalog-config.cmake DESTINATION share/datalog)
install(EXPORT datalog DESTINATION share/datalog)

if(WITH_TESTS)
file(GLOB_RECURSE datalog_testlib_src src/test/native/include/*.h)
add_library(datalog_testlib INTERFACE ${datalog_test_src})
target_include_directories(datalog_testlib INTERFACE src/test/native/include)

wpilib_add_test(datalog src/test/native/cpp)
target_link_libraries(datalog_test datalog googletest datalog_testlib wpiutil)
if(MSVC)
target_compile_options(datalog_test PRIVATE /utf-8)
endif()
endif()
32 changes: 32 additions & 0 deletions datalog/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
ext {
useJava = true
useCpp = true
baseId = 'datalog'
groupId = 'edu.wpi.first.datalog'

nativeName = 'datalog'
devMain = 'edu.wpi.first.datalog.DevMain'
}

apply from: "${rootDir}/shared/jni/setupBuild.gradle"

nativeUtils.exportsConfigs {
datalog {
}
}

model {
components {
all {
it.sources.each {
it.exportedHeaders {
srcDirs 'src/main/native/include'
}
}
}
}
}

dependencies {
api project(":wpiutil")
}
5 changes: 5 additions & 0 deletions datalog/datalog-config.cmake.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
@FILENAME_DEP_REPLACE@
include(${SELF_DIR}/wpiutil.cmake)
if(@WITH_JAVA@)
include(${SELF_DIR}/wpiutil_jar.cmake)
endif()
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,10 @@
#include <fmt/chrono.h>
#include <fmt/format.h>
#include <fmt/ranges.h>

#include "wpi/DataLogReader.h"
#include "wpi/DenseMap.h"
#include "wpi/MemoryBuffer.h"
#include "wpi/print.h"
#include <wpi/DenseMap.h>
#include <wpi/MemoryBuffer.h>
#include <wpi/datalog/DataLogReader.h>
#include <wpi/print.h>

int main(int argc, const char** argv) {
if (argc != 2) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
#include <utility>
#include <vector>

#include "wpi/DataLogBackgroundWriter.h"
#include "wpi/print.h"
#include <wpi/datalog/DataLogBackgroundWriter.h>
#include <wpi/print.h>

int main(int argc, char** argv) {
using std::chrono::duration_cast;
Expand Down
12 changes: 12 additions & 0 deletions datalog/src/dev/java/edu/wpi/first/datalog/DevMain.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// Copyright (c) FIRST and other WPILib contributors.
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

package edu.wpi.first.datalog;

public final class DevMain {
/** Main entry point. */
public static void main(String[] args) {}

private DevMain() {}
}
5 changes: 5 additions & 0 deletions datalog/src/dev/native/cpp/main.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// Copyright (c) FIRST and other WPILib contributors.
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

int main() {}
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

package edu.wpi.first.util.datalog;
package edu.wpi.first.datalog;

import java.util.Arrays;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

package edu.wpi.first.util.datalog;
package edu.wpi.first.datalog;

/** Log boolean values. */
public class BooleanLogEntry extends DataLogEntry {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

package edu.wpi.first.util.datalog;
package edu.wpi.first.datalog;

import edu.wpi.first.util.WPIUtilJNI;
import edu.wpi.first.util.protobuf.Protobuf;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

package edu.wpi.first.util.datalog;
package edu.wpi.first.datalog;

/**
* A data log background writer that periodically flushes the data log on a background thread. The
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

package edu.wpi.first.util.datalog;
package edu.wpi.first.datalog;

/** Log entry base class. */
public class DataLogEntry {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// Open Source Software; you can modify and/or share it under the terms of
// the WPILib BSD license file in the root directory of this project.

package edu.wpi.first.util.datalog;
package edu.wpi.first.datalog;

import java.util.Iterator;
import java.util.NoSuchElementException;
Expand Down
Loading
Loading