db_benchmark_test.go | |
---|---|
|
|
Benchmark for storage backend - the Postgres or RDS Please note that database schema needs to be prepared before benchmarks and needs to be migrated to latest version. Look into README.md for detailed information how to check DB status and how to migrate the database. Additional info about benchmarks can be found at https://redhatinsights.github.io/ccx-notification-writer/benchmarks.html Benchmarks use reports represented as JSON files that are stored in tests/reports/ subdirectory. Documentation in literate-programming-style is available at: https://redhatinsights.github.io/ccx-notification-writer/packages/dbbenchmarktest.html |
|
Configuration-related constants |
|
environment variable with name of configuration file (without extension) |
|
name of configuration file (without extension) |
|
directory containing reports used by benchmarks |
|
JSON files containing reports |
|
SQL statements Table reported V1 is reported table without eventtype column and constraint for this column Table reported V2 is reported table with eventtype column and constraint for this column |
|
SQL statement to drop tables used by benchmarks |
|
reported table without fkeventtype constraint |
|
reported table with fkeventtype constraint |
|
Index for the reported table used in benchmarks for notified_at column |
|
Index for the reported table used in benchmarks for notified_at column |
|
Index for the reported table for updatedatdesc column |
|
Index for the reported table for updatedatdesc column |
|
Optional index for the reported table for eventtypeid column. Index type is set to BTree. (https://www.postgresql.org/docs/current/indexes-types.html for more info) |
|
Optional index for the reported table for eventtypeid column. Index type is set to hash. (https://www.postgresql.org/docs/current/indexes-types.html for more info) |
|
Optional index for the reported table column. Index type is set to BRIN. (https://www.postgresql.org/docs/current/indexes-types.html for more info) |
|
Insert one record into reported table w/o eventtypeid column |
|
Insert one record into reported table with eventtypeid column |
|
SQL query used to display older records from reported table |
|
SQL query used to display older records from reported table |
|
SQL query used to display older records from reported table |
|
SQL query used to display older records from reported table |
|
insertIntoReportedFunc type represents any function to be called to insert records into reported table |
|
CLI flags that can be provided (please note that in tests it is NOT NEEDED to parse flags) |
|
initLogging function initializes logging that's used internally by functions from github.com/RedHatInsights/ccx-notification-writer package |
|
loadConfiguration function loads configuration prepared to be used by benchmarks |
|
initializeStorage function initializes storage and perform connection to database |
|
perform storage initialization |
|
for benchmarks, we just need to have connection to DB, not the whole Storage structure |
|
check if connection has been established |
|
connection should be established at this moment |
|
unfortunately there seems to be no other way how to handle per-benchmark setup properly than to use global variables |
|
setup function performs all DB initializations needed for DB benchmarks. Can be called from any benchmark. In case of any error detected, benchmarks fail immediately. |
|
next time, benchmarks will use already established connection to database |
|
execSQLStatement function executes any provided SQL statement. In case of any error detected, benchmarks fail immediately. |
|
check for any error, possible to exit immediately |
|
insertIntoReportedV1Statement function inserts one new record into reported table v1. In case of any error detected, benchmarks fail immediately. |
|
following columns needs to be updated with data: 1 | orgid | integer | not null | 2 | accountnumber | integer | not null | 3 | cluster | character(36) | not null | 4 | notificationtype | integer | not null | 5 | state | integer | not null | 6 | report | character varying | not null | 7 | updatedat | timestamp without time zone | not null | 8 | notifiedat | timestamp without time zone | not null | 9 | errorlog | character varying | | |
|
perform insert |
|
check for any error, possible to exit immediately |
|
insertIntoReportedV2Statement function inserts one new record into reported table v2. In case of any error detected, benchmarks fail immediately. |
|
following columns needs to be updated with data: 1 | orgid | integer | not null | 2 | accountnumber | integer | not null | 3 | cluster | character(36) | not null | 4 | notificationtype | integer | not null | 5 | state | integer | not null | 6 | report | character varying | not null | 7 | updatedat | timestamp without time zone | not null | 8 | notifiedat | timestamp without time zone | not null | 9 | errorlog | character varying | | 10| eventtypeid | integer | | |
|
perform insert |
|
check for any error, possible to exit immediately |
|
runBenchmarkInsertIntoReportedTable function perform several inserts into reported table v1 or v2 (depending on injected function). In case of any error detected, benchmarks fail immediately. |
|
retrieve DB connection |
|
run all init SQL statements |
|
good citizens cleanup properly defer execSQLStatement(b, connection, dropTableReportedV1) |
|
time to start benchmark |
|
perform DB benchmark |
|
runBenchmarkSelectOrDeleteFromReportedTable function perform several inserts into reported table v1 or v2 (depending on injected function) and the run queries or delete statements against such table. In case of any error detected, benchmarks fail immediately. |
|
retrieve DB connection |
|
run all init SQL statements |
|
good citizens cleanup properly defer execSQLStatement(b, connection, dropTableReportedV1) |
|
fill-in the table (no part of benchmark, so don't measure time there) |
|
time to start benchmark |
|
perform DB benchmark |
|
perform benchmarks for SELECT statement (query) |
|
perform benchmarks for DELETE statement |
|
fill-in the table again (no part of benchmark, so don't measure time there) |
|
readReport function tries to read report from file. Benchmark will fail in any error. |
|
getIndicesForReportedTableV1 is helper function to return map with all possible indices combinations |
|
getIndicesForReportedTableV2 is helper function to return map with all possible indices combinations |
|
benchmarkInsertReportsIntoReportedTableImpl is an implementation of benchmark to insert reports into reported table with or without eventtypeid column. Table with all possible indices combination is tested. |
|
try all indices combinations |
|
new benchmark |
|
prepare all SQL statements to be run before benchmark |
|
add all init statements |
|
add all statements to create indices |
|
now everything's ready -> run benchmark |
|
benchmarkSelectOrDeleteOldReportsFromReportedTableImpl is an implementation of benchmark to query reports from reported or delete reports from such table with or without eventtypeid column. Table with all possible indices combination is tested. |
|
try all indices combinations |
|
benchmark with various reports count stored in table |
|
new benchmark |
|
prepare all SQL statements to be run before benchmark |
|
add all init statements |
|
add all statements to create indices |
|
now everything's ready -> run benchmark |
|
computeLength function calculates correct length of slice with values in range <min, max) with given step |
|
poor man's integer ceiling |
|
parseReportsCount tries to parse -reports-count parameter that have following format: -reports-count=1,2,5 nolint:prealloc |
|
split the string first |
|
now try to parse parts sequentially |
|
skip problematic part |
|
append parsed value |
|
readPossibleReportsCount read sequence of # reports to be inserted into tested table. It is based on following CLI flags: -min-reports=1 -max-reports=10 -reports-step=2 -reports-count=1,2,5 |
|
check if CLI flags are specified |
|
make slice to store all reports count |
|
fill-in the slice with correct values |
|
BenchmarkInsertReportsIntoReportedTableV1 checks the speed of inserting into reported table without event_type column |
|
try to insert empty reports |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertReportsIntoReportedTableV2 checks the speed of inserting into reported table with event_type column |
|
try to insert empty reports |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table with eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertAnalysisMetadataOnlyReportIntoReportedTableV1 checks the speed of inserting into reported table without event_type column |
|
report with size approx 0.5kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertAnalysisMetadataOnlyReportIntoReportedTableV2 checks the speed of inserting into reported table with event_type column |
|
report with size approx 0.5kB |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table with eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertSmallReportIntoReportedTableV1 checks the speed of inserting into reported table without event_type column |
|
report with size approx 2kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertSmallReportIntoReportedTableV2 checks the speed of inserting into reported table with event_type column |
|
report with size approx 2kB |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table with eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertMiddleReportIntoReportedTableV1 checks the speed of inserting into reported table without event_type column |
|
report with size approx 4kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertMiddleReportIntoReportedTableV2 checks the speed of inserting into reported table with event_type column |
|
report with size approx 4kB |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table with eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertLargeReportIntoReportedTableV1 checks the speed of inserting into reported table without event_type column |
|
report with size over 8kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkInsertLargeReportIntoReportedTableV2 checks the speed of inserting into reported table with event_type column |
|
report with size over 8kB |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table with eventtypeid column |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldEmptyRecordsFromReportedTableV1 tests the query to reported table used inside CCX Notification Service |
|
empty report |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldEmptyRecordsFromReportedTableV2 tests the query to reported table used inside CCX Notification Service |
|
empty report |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldSmallRecordsFromReportedTableV1 tests the query to reported table used inside CCX Notification Service |
|
report with size over 2kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldSmallRecordsFromReportedTableV2 tests the query to reported table used inside CCX Notification Service |
|
report with size over 2kB |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldMiddleRecordsFromReportedTableV1 tests the query to reported table used inside CCX Notification Service |
|
report with size over 4kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldMiddleRecordsFromReportedTableV2 tests the query to reported table used inside CCX Notification Service |
|
report with size over 4kB |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldLargeRecordsFromReportedTableV1 tests the query to reported table used inside CCX Notification Service |
|
report with size over 8kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkSelectOldLargeRecordsFromReportedTableV2 tests the query to reported table used inside CCX Notification Service |
|
report with size over 8kB |
|
default init statements for reported table with eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldEmptyRecordsFromReportedTableV1 tests the deletion statement from reported table used inside CCX Notification Service |
|
empty report |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldEmptyRecordsFromReportedTableV2 tests the deletion statement from reported table used inside CCX Notification Service |
|
empty report |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldSmallRecordsFromReportedTableV1 tests the deletion statement from reported table used inside CCX Notification Service |
|
report with size over 2kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldSmallRecordsFromReportedTableV2 tests the deletion statement from reported table used inside CCX Notification Service |
|
report with size over 2kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldMiddleRecordsFromReportedTableV1 tests the deletion statement from reported table used inside CCX Notification Service |
|
report with size over 4kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldMiddleRecordsFromReportedTableV2 tests the deletion statement from reported table used inside CCX Notification Service |
|
report with size over 4kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldLargeRecordsFromReportedTableV1 tests the deletion statement from reported table used inside CCX Notification Service |
|
report with size over 8kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|
BenchmarkDeleteOldLargeRecordsFromReportedTableV2 tests the deletion statement from reported table used inside CCX Notification Service |
|
report with size over 8kB |
|
default init statements for reported table without eventtypeid column |
|
all possible indices combinatiors for reported table without eventtypeid column |
|
read number of reports to be inserted into the table under tests |
|
run benchmarks with various combination of indices |
|