Skip to content

Commit f813f4c

Browse files
committed
Implement RW interface for MySQL
1 parent 2ab7a96 commit f813f4c

File tree

14,403 files changed

+7184267
-1
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

14,403 files changed

+7184267
-1
lines changed

.DS_Store

4 KB
Binary file not shown.

README.md

+41
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,46 @@
22

33
## RW
44

5+
**RW** is **an intuitive and explicit storage command to address read stalls**. Upon read stall, both read and write can be requested to the flash storage in a single call. And the read can be processed as soon as the dirty page is copied to the storage buffer cache using DMA, without waiting for the NAND write to finish. By replacing two I/O calls for write and read upon read stalls with one call, RW can simplify the structure of the buffer manager in DBMS and reduce the number of I/O interrupts and consequently the number of context switches.
6+
7+
### Structure
8+
9+
The major modifications to implement RW were made in the directories below:
10+
11+
- [`storage/innobase/buf`](storage/innobase/buf): The database buffer implementation for InnoDB
12+
- [`storage/innobase/fil`](storage/innobase/fil): File I/O operations for InnoDB
13+
- [`storage/innobase/include`](storage/innobase/include): The collection of header files for InnoDB
14+
15+
RW consists of the following components:
16+
17+
- Basic modules for the buffer manager to utilize the RW command (e.g., buffer initialization, buffer allocation, buffer replacement, etc.)
18+
- [`buf0buf.cc`](storage/innobase/buf/buf0buf.cc)
19+
- The module for proceeding the transaction immediately using the read data buffer without a separate free buffer acquisition procedure
20+
- [`buf0rea.cc`](storage/innobase/buf/buf0rea.cc)
21+
- [`buf0lru.cc`](storage/innobase/buf/buf0lru.cc)
22+
- The module for sending the RW NVM command set to storage
23+
- [`rw0rw.cc`](storage/innobase/rw/rw0rw.cc), [`rw0rw.h`](storage/innobase/include/rw0rw.h)
24+
- [`buf0flu.cc`](storage/innobase/buf/buf0flu.cc)
25+
- [`fil0fil.cc`](storage/innobase/fil/fil0fil.cc)
26+
- [`os0file.cc`](storage/innobase/os/os0file.cc)
27+
28+
You can check the modified code by searching for the `#ifdef RW_CMD` keyword in the file. For example:
29+
30+
> `storage/innobase/fil/fil0fil.cc`
31+
32+
```cpp
33+
#ifdef RW_CMD
34+
...
35+
fil_mutex_enter_and_prepare_for_rw_io(
36+
/*===============================*/
37+
ulint space_id, /*!< in: space id to write */
38+
ulint space_id_to_read /*!< in: space id to read */
39+
)
40+
{
41+
fil_space_t* space;
42+
...
43+
#endif /* RW_CMD */
44+
```
45+
546
## R-Buf
647

cosmos-firmware/ftl_config.h

+1-1
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@
183183

184184
//************************************************************************
185185
#define BITS_PER_FLASH_CELL SLC_MODE //user configurable factor
186-
#define USER_BLOCKS_PER_LUN 142 //user configurable factor //mijin 4096 240
186+
#define USER_BLOCKS_PER_LUN 142 //user configurable factor e.g., 4096, 240 (current = 32GB)
187187
#define USER_CHANNELS (NUMBER_OF_CONNECTED_CHANNEL) //user configurable factor
188188
#define USER_WAYS 8 //user configurable factor
189189
//************************************************************************

mysql/BUILD-CMAKE

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
The latest information about building MySQL with CMake is located on
2+
http://dev.mysql.com/doc/internals/en/cmake.html
3+
4+
See also BUILD/README

mysql/BUILD/README

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
This directory used to contain lots of build scripts for building
2+
MySQL on various platforms. They are mostly gone, use cmake instead.
3+
4+
Some scripts have been kept for backward compatibility (other scripts
5+
depend on them). Others have been kept to illustrate how to enable
6+
gcov or gprof, or explicitly selecting 64bit architecture. If you
7+
want to use these scripts for building, you may want to speed things
8+
up a bit by setting environment variable MAKEFLAGS="-j8"
9+
(or some other number/flag depending on your environment)
10+
11+
The recommended way to build MySQL for developers:
12+
13+
cd <some build directory>
14+
cmake <path to source directory>
15+
make
16+
17+
This will give you a release build, with compiler options taken from
18+
../cmake/build_configurations/compiler_options.cmake
19+
and "feature set" taken from
20+
../cmake/build_configurations/feature_set.cmake
21+
22+
Adding -DWITH_DEBUG=1 to the cmake command line gives you a debug build.
23+
24+
25+
Building on Windows is slightly different:
26+
cd <some build directory>
27+
cmake <path to source directory>
28+
29+
cmake --build . --config Debug
30+
or
31+
cmake --build . --config RelWithDebInfo
32+
33+
34+
If you have special needs, you can disable the defaults by setting
35+
these cmake variables off:
36+
37+
WITH_DEFAULT_COMPILER_OPTIONS
38+
WITH_DEFAULT_FEATURE_SET
39+
40+
Note: For building with unit tests, you need to tell cmake where to find
41+
the sources, see ../unittest/gunit/CMakeLists.txt
42+
You should *not* 'make install' googletest/googlemock on your system,
43+
the libraries must be built from source, with the unit tests.

mysql/BUILD/SETUP.sh

+258
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,258 @@
1+
#!/bin/sh
2+
3+
# Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
4+
#
5+
# This program is free software; you can redistribute it and/or
6+
# modify it under the terms of the GNU Library General Public
7+
# License as published by the Free Software Foundation; version 2
8+
# of the License.
9+
#
10+
# This program is distributed in the hope that it will be useful,
11+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
12+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
13+
# Library General Public License for more details.
14+
#
15+
# You should have received a copy of the GNU Library General Public
16+
# License along with this library; if not, write to the Free
17+
# Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
18+
# MA 02110-1301, USA
19+
20+
########################################################################
21+
22+
get_key_value()
23+
{
24+
echo "$1" | sed 's/^--[a-zA-Z_-]*=//'
25+
}
26+
27+
usage()
28+
{
29+
cat <<EOF
30+
Usage: $0 [-h|-n] [configure-options]
31+
-h, --help Show this help message.
32+
-n, --just-print Don't actually run any commands; just print them.
33+
-c, --just-configure Stop after running configure.
34+
--with-debug=full Build with full debug(no optimizations, keep call stack).
35+
--warning-mode=[old|pedantic|maintainer]
36+
Influences the debug flags. Old is default.
37+
--prefix=path Build with prefix 'path'.
38+
39+
Note: this script is intended for internal use by MySQL developers.
40+
EOF
41+
}
42+
43+
parse_options()
44+
{
45+
while test $# -gt 0
46+
do
47+
case "$1" in
48+
--prefix=*)
49+
prefix=`get_key_value "$1"`;;
50+
--with-debug=full)
51+
full_debug="=full";;
52+
--warning-mode=*)
53+
warning_mode=`get_key_value "$1"`;;
54+
-c | --just-configure)
55+
just_configure=1;;
56+
-n | --just-print | --print)
57+
just_print=1;;
58+
-h | --help)
59+
usage
60+
exit 0;;
61+
*)
62+
echo "Unknown option '$1'"
63+
exit 1;;
64+
esac
65+
shift
66+
done
67+
}
68+
69+
########################################################################
70+
71+
if test ! -f sql/mysqld.cc
72+
then
73+
echo "You must run this script from the MySQL top-level directory"
74+
exit 1
75+
fi
76+
77+
prefix="/usr/local/mysql"
78+
just_print=
79+
just_configure=
80+
warning_mode=
81+
maintainer_mode=
82+
full_debug=
83+
84+
parse_options "$@"
85+
86+
if test -n "$MYSQL_BUILD_PREFIX"
87+
then
88+
prefix="$MYSQL_BUILD_PREFIX"
89+
fi
90+
91+
set -e
92+
93+
#
94+
# Check for the CPU and set up CPU specific flags. We may reset them
95+
# later.
96+
#
97+
path=`dirname $0`
98+
. "$path/check-cpu"
99+
100+
export AM_MAKEFLAGS
101+
AM_MAKEFLAGS="-j 6"
102+
103+
# SSL library to use.--with-ssl will select our bundled yaSSL
104+
# implementation of SSL. To use openSSl you will nee too point out
105+
# the location of openSSL headers and lbs on your system.
106+
# Ex --with-ssl=/usr
107+
SSL_LIBRARY=--with-ssl
108+
109+
if [ "x$warning_mode" = "xpedantic" ]; then
110+
warnings="-W -Wall -ansi -pedantic -Wno-long-long -Wno-unused -D_POSIX_SOURCE"
111+
c_warnings="$warnings"
112+
cxx_warnings="$warnings -std=c++98"
113+
# NOTE: warning mode should not influence optimize/debug mode.
114+
# Please feel free to add a separate option if you don't feel it's an overkill.
115+
debug_extra_cflags="-O0"
116+
# Reset CPU flags (-mtune), they don't work in -pedantic mode
117+
check_cpu_cflags=""
118+
elif [ "x$warning_mode" = "xmaintainer" ]; then
119+
c_warnings="-Wall -Wextra"
120+
cxx_warnings="$c_warnings -Wno-unused-parameter"
121+
maintainer_mode="--enable-mysql-maintainer-mode"
122+
debug_extra_cflags="-g3"
123+
else
124+
# Both C and C++ warnings
125+
warnings="-Wall -Wextra -Wunused -Wwrite-strings"
126+
127+
# For more warnings, uncomment the following line
128+
# warnings="$warnings -Wshadow"
129+
130+
# C warnings
131+
c_warnings="$warnings"
132+
# C++ warnings
133+
cxx_warnings="$warnings -Wno-unused-parameter"
134+
# cxx_warnings="$cxx_warnings -Woverloaded-virtual -Wsign-promo"
135+
cxx_warnings="$cxx_warnings -Wnon-virtual-dtor"
136+
debug_extra_cflags="-O0 -g3 -gdwarf-2"
137+
fi
138+
139+
# Set flags for various build configurations.
140+
# Used in -valgrind builds
141+
# Override -DFORCE_INIT_OF_VARS from debug_cflags. It enables the macro
142+
# LINT_INIT(), which is only useful for silencing spurious warnings
143+
# of static analysis tools. We want LINT_INIT() to be a no-op in Valgrind.
144+
valgrind_flags="-UFORCE_INIT_OF_VARS -DHAVE_purify "
145+
valgrind_flags="$valgrind_flags -DMYSQL_SERVER_SUFFIX=-valgrind-max"
146+
valgrind_configs="--with-valgrind"
147+
#
148+
# Used in -debug builds
149+
debug_cflags="-DEXTRA_DEBUG -DFORCE_INIT_OF_VARS "
150+
debug_cflags="$debug_cflags -DSAFE_MUTEX"
151+
error_inject="--with-error-inject "
152+
#
153+
# Base C++ flags for all builds
154+
base_cxxflags="-felide-constructors"
155+
#
156+
# Flags for optimizing builds.
157+
# Be as fast as we can be without losing our ability to backtrace.
158+
fast_cflags="-O3 -fno-omit-frame-pointer"
159+
160+
debug_configs="--with-debug"
161+
if [ -z "$full_debug" ]
162+
then
163+
debug_cflags="$debug_cflags $debug_extra_cflags"
164+
fi
165+
166+
167+
#
168+
# Configuration options.
169+
#
170+
base_configs="--prefix=$prefix --enable-assembler "
171+
base_configs="$base_configs --with-extra-charsets=complex "
172+
base_configs="$base_configs --enable-thread-safe-client "
173+
base_configs="$base_configs --with-big-tables $maintainer_mode"
174+
175+
176+
if test -d "$path/../cmd-line-utils/libedit"
177+
then
178+
base_configs="$base_configs --with-libedit"
179+
fi
180+
181+
static_link="--with-mysqld-ldflags=-all-static "
182+
static_link="$static_link --with-client-ldflags=-all-static"
183+
# we need local-infile in all binaries for rpl000001
184+
# if you need to disable local-infile in the client, write a build script
185+
# and unset local_infile_configs
186+
local_infile_configs="--enable-local-infile"
187+
188+
189+
max_no_embedded_configs="$SSL_LIBRARY --with-plugins=max"
190+
max_no_ndb_configs="$SSL_LIBRARY --with-plugins=max-no-ndb --with-embedded-server"
191+
max_configs="$SSL_LIBRARY --with-plugins=max --with-embedded-server"
192+
193+
#
194+
# CPU and platform specific compilation flags.
195+
#
196+
alpha_cflags="$check_cpu_cflags -Wa,-m$cpu_flag"
197+
amd64_cflags="$check_cpu_cflags"
198+
amd64_cxxflags="" # If dropping '--with-big-tables', add here "-DBIG_TABLES"
199+
pentium_cflags="$check_cpu_cflags"
200+
pentium64_cflags="$check_cpu_cflags -m64"
201+
ppc_cflags="$check_cpu_cflags"
202+
sparc_cflags=""
203+
204+
if gmake --version > /dev/null 2>&1
205+
then
206+
make=gmake
207+
else
208+
make=make
209+
fi
210+
211+
if test -z "$CC" ; then
212+
CC=gcc
213+
fi
214+
215+
if test -z "$CXX" ; then
216+
CXX=g++
217+
fi
218+
219+
# If ccache (a compiler cache which reduces build time)
220+
# (http://samba.org/ccache) is installed, use it.
221+
# We use 'grep' and hope 'grep' will work as expected
222+
# (returns 0 if finds lines)
223+
if test "$USING_GCOV" != "1"
224+
then
225+
# Not using gcov; Safe to use ccache
226+
CCACHE_GCOV_VERSION_ENABLED=1
227+
fi
228+
229+
if ccache -V > /dev/null 2>&1 && test "$CCACHE_GCOV_VERSION_ENABLED" = "1"
230+
then
231+
echo "$CC" | grep "ccache" > /dev/null || CC="ccache $CC"
232+
echo "$CXX" | grep "ccache" > /dev/null || CXX="ccache $CXX"
233+
fi
234+
235+
# gcov
236+
237+
# The -fprofile-arcs and -ftest-coverage options cause GCC to instrument the
238+
# code with profiling information used by gcov.
239+
# The -DDISABLE_TAO_ASM is needed to avoid build failures in Yassl.
240+
# The -DHAVE_gcov enables code to write out coverage info even when crashing.
241+
242+
gcov_compile_flags="-fprofile-arcs -ftest-coverage"
243+
gcov_compile_flags="$gcov_compile_flags -DDISABLE_TAO_ASM"
244+
gcov_compile_flags="$gcov_compile_flags -DMYSQL_SERVER_SUFFIX=-gcov -DHAVE_gcov"
245+
246+
# GCC4 needs -fprofile-arcs -ftest-coverage on the linker command line (as well
247+
# as on the compiler command line), and this requires setting LDFLAGS for BDB.
248+
249+
gcov_link_flags="-fprofile-arcs -ftest-coverage"
250+
251+
gcov_configs="--with-gcov"
252+
253+
# gprof
254+
255+
gprof_compile_flags="-O2 -pg -g"
256+
257+
gprof_link_flags="--disable-shared $static_link"
258+

mysql/BUILD/autorun.sh

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
#!/bin/sh
2+
3+
# Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
4+
#
5+
# This program is free software; you can redistribute it and/or
6+
# modify it under the terms of the GNU Library General Public
7+
# License as published by the Free Software Foundation; version 2
8+
# of the License.
9+
#
10+
# This program is distributed in the hope that it will be useful,
11+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
12+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
13+
# Library General Public License for more details.
14+
#
15+
# You should have received a copy of the GNU Library General Public
16+
# License along with this library; if not, write to the Free
17+
# Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston,
18+
# MA 02110-1301, USA
19+
20+
# Create MySQL cmake configure wrapper
21+
22+
die() { echo "$@"; exit 1; }
23+
24+
# Use a configure script that will call CMake.
25+
path=`dirname $0`
26+
cp $path/cmake_configure.sh $path/../configure
27+
chmod +x $path/../configure

0 commit comments

Comments
 (0)