snowplow / snowplow-rdb-loader

52%

master: 52%

LAST BUILD ON BRANCH refs/tags/6.1.1-rc6

branch: refs/tags/6.1.1-rc6

CHANGE BRANCH

x

Reset

• refs/tags/6.1.1-rc6
• 05100bfda60f7930aa488ef86797f8e057b2b038
• 05e396b0152b9b2a520ea7e97c840f07ae5813cc
• 067e98c7a0f0e9add6384039920a12834366ea17
• 07125c657f69e8ee67e01a602d0916b688f21ea7
• 07675034c25d1e2147722219a94d600127749b24
• 07a8c77b4a3f7b7b6a96e769baad0d6b778b95a1
• 090423aa15f7c39a1c90ba588ed7bc83a75d55a2
• 0b8e797c9f9e9e8d31417cb6730a46fef9cea430
• 0e81a87a5c216f3333bed2f6cc8940ebe243903f
• 112305887a18708c3cec1d9cac8bb30412efa150
• 115fa08fc218b6022d6142bf320922de9314e0df
• 13593546210c42cf8c48f936e64155affbac4ab4
• 15687773075e59d4a02e992d819c93fb484ee711
• 16d3b89b5c89b4d43d9e312f51cbc5c90c464d27
• 189190eaf192680be9cdbc6254603eb4a85c2a6d
• 19d7c754ccb9ba154f9cc118a9c5bb8cb62fa318
• 1de8918fedfa77bcf483ea87589a6d020384adab
• 1df9e3a1d02c8a3edc59e6e8a6384000f04a8f9f
• 1e0431660a146c399b07c4a1d4c6f8360bcb56e8
• 1e06b0abb0751ad31a9622ab8e6f923f37d70b24
• 1e574c0048d73f85f6c0aa21b012fcedcf3c1322
• 2134bc704c5abfe38a0aef7c2cd08d7a6ff824a1
• 22802ffcf262a03656dadaa5de2ce1c584bffaaf
• 23ecb3c485e07512cccb99913aa05e630223b491
• 258de509854e89f92b5fbb05a417901937153f11
• 26330f297d2402b71413456d32ce19921b082fdc
• 2a457e388e040bbc1c66d4d86f708a740b095274
• 2aa7bb83c75e928d034138a882772f3a4656f44e
• 2bf4aff9087a532f8d5327557792fd729a90ca01
• 2c490b12452fe38b1b566423702aa97c9c5ac235
• 2c745cdbd8dc9c9eaeb78fc0554e653a51b6c1b6
• 2dd60716d14454af55289a6cefab85d974c22f21
• 30fb39138a4046d55f3baadff400964624d18e1d
• 31e6f4e469e30637fe401a7efef80f92e72f4a61
• 336c097b17b90a252ea2e7f1e01adc7c42b4abe8
• 34cb85ca4d89ea9e2479a5301f73f734f4f780eb
• 3549f577fc00a31f5a1302db74955590487760b7
• 380b48b26c14043909cc5b3bccdf812c3f47243a
• 395c8b180f30c7bda93cee0c7a78698543190d3e
• 3a7ec862a88d9904b5142e2729a134d3903c1512
• 3a80162d15261d4cf4a875039b3c5a5990091f19
• 3acd7070eb06fb095bce641450e8eba91bda5397
• 3b4e1eb9bdf8195b35ed177bb5d3626bccd878e7
• 3c22f3d0a92ebd723b8cff0bf33823d7d0b2fbc8
• 3dbc18182796f4f0b3e359b0125f75f57ab23c1a
• 3f04338116523ac9e0ff3a834f1657b296d4460f
• 3f5cec8dbbba61bfc49271db88ec20305fe20f65
• 406e64f50407397ca1f938901df9aec5adcd9050
• 4095c619ced60e878babf46ff2e25e04a1093217
• 4456a36a925ec566f049bd314d390bf0931b9b0b
• 44dd455febd87a6cbb03865fb5733f4f71301a91
• 45fa5415a0ce4cfabacf6bcd17ab6d6508abb098
• 47a8648a5a48902e23dfb009ed1e87536039581e
• 48a00af134bc40595c93f6db60b2fd6f1be65c22
• 4b234bebd544e2b7a45f04a3a3fb49a656e74c20
• 4b2be9e05ff2874fc4d9b73a719437b877b79fdc
• 4c5fee0163ffe110ce3940d6e9de8424964ffecb
• 509c91095a05b5fb7b3f14a2abf8ce5f25a3ad6f
• 51554159d77eba9794e4bef2cd9269c1c76c2dfe
• 51e2eb3c0f1def889eb9d87be4ae5b9f35c81ddc
• 553d0b6cf4c096b4eb88bdc00dab3e6ef4f9c94c
• 57c258de2853033663b705f65589c84aae8f5fba
• 57d8f9a4d2435d0df7929c00cd5873e3c8ff0c6d
• 582bb108b8d51a60b40ae6a3352318efcb02ac4a
• 58fe404e62870f3c692419f8d3eb90f51980c55e
• 59736a182af580d8a72c91d39bcfada0dad13919
• 5a41eb5bd43d7976d49209900cae4c3b15a8dcf1
• 5bd4eef35f42a6aba45115d12ef8d310a3955f1b
• 5e911fd35e86c8d4381c4c758e1bc23d811d4637
• 5f02a0f317cc45dc21b706e97b8bf81daab8880e
• 60dba84ff623596405821b229d5e74a2640e7c5b
• 6230261e020cb66d0ef5730d6b1d0bb1b8f28aa7
• 63c4a1b0fd4fba1710681a25a6940cd4d041cbe7
• 6491b3be46fef8bbc75b54343b1d34de22968194
• 6550694384515e96c69d58c2ce1acaad66491b4e
• 6a996ecd3e6cbaa5d0edcbb5d82fc3a30cd34eca
• 6ce377a628b10231e2c3d93f22038d8c78f9755c
• 6fd173a8065aad33d64a4659835642c842c1053c
• 70cfa26f27c08805fee735400e58e3b29f09cbb8
• 72f765ecf200926c46cdcabe5d224ca8d4187545
• 73b308e59a1f809f119255ab85be6ff3d300cbf6
• 755195a30b4141ebc70132a1c146e4cdefce9b30
• 79e4da646da109907df21530dcfd3f778f7404d1
• 7a5354087fb2a52bd86ad7f087754cba9e2ae528
• 7af2175bf3b32b014d7060c79b19d443ba1f5fd0
• 7b492921d434e6f081d990612321be502eeabd06
• 7c0460a4fcac7e3623f4a282c1f1f9bdb6148223
• 7ed039ea80586cc8c73d7ccc61379037bedfa069
• 823493c4eed8d701d16a9d4f7e0c8a80bf8d65b6
• 8387a0ce818b09725f9ad45ef50a0c746f76f591
• 847d029b1aae83b3053263b2204ef5de0c813c08
• 85adaa0df4e061a9c909feb5b279afb0f5ae41fb
• 8808a886ae917f558b90a387644df033129b673f
• 892cb8ebddcd14e6096c8e032e57def30fb039bc
• 8a663e282b375dfd4afb7fde898d9bb2c5c54ab5
• 8d0cc76c6bb784469ec3b3430cae46235c0db832
• 9153db758429854af2728caacafa74de8895258a
• 957a9627c3cd09369df2a8f5cfe59afdda213c2a
• 95b70d822977ce4ab7ab699e05409dbdb6ea99f2
• 95dd7f4a17a58aa614517dc595a98759193dc394
• 9d28f07b04491e629a6785a7f5490f6fcb17b43d
• a0982968d27af746114a4e8cbacb5d3a1d080286
• a279338be8c172eaec585e7dc747eb4fae9b3c1c
• a294bce5981d9bdfb72c6465563fdff6ccec81da
• a2ce17aa5d3f15bbd97fdb64e8c3ea4d5380d6e2
• a372c23d6cf2290a92e950de11d8ddaaad05e0c9
• a46acd9b15cbe1ae4b44ce48102d85da804ce206
• a54a1492433a4f6d1fb1a6b36a9a1e89aaa48ad8
• a67093a43ae53db70c25e7b617c02ab49c773fef
• a69f940118ccfbd8586ffb7f0afe6b8d0c3a8104
• a732fe98e20548233c246bf6e93ed4191c43129a
• a7842636f35b2e799393345813d8e9171933f10b
• a8425a5b7aac9fc2862cb55dc80a69e4223f8ca2
• abebe3c163257cf2b96ea4feb24d836942a643b8
• ad4e76fd5cf13d19df87e4df99ecb3a23562049b
• ad7e781e3eccf6e0bc29180fd96c066f85b8e6dc
• adbda17936f83f1b7a9b7c664f5f039091264c2b
• add-sentry
• af39f23e840255d88d37188ebc94e0c1afd4a3d7
• b049727a634db11dbf58f5308c20e4036ac44f5e
• b1f5ea7c79ee539140a2649e33f78138d0685f5c
• b50c0e554137679241383795c8c5560e2cd6a28f
• b5c2e3136f6427153c72fb39c43bcbdc1ab39645
• b60fd6bfc363b1c21fcd8c8ea56764f47e40b94f
• b64b63673e0ce854bd85abc6808907b86a51ed20
• b7ba802acea77daab3b3e1f884f3d9ad2a832556
• b98e13c94e65c7feea17940810cbc8b361d39c29
• ba506f1b951c670d690f35f4c58a82fcc4e5d9ae
• bda8105b12da0a4f7cd2a5a855806cc808054142
• be10a18d4a7f0bd0c16ba9974a0578776ef3c951
• bea2034d9e951321f3395e0492627ff783e4370d
• beecd1325e8e6b5e53ec9085bdebcbe7bfc364be
• bf33a00082f767c71351d243a733005921c32e4e
• bfb5bf24bd68710caf9d9fb6def17f82dd3bcb87
• bump-dependencies-6.1.3
• c40990d5d81ce9360c9f9c37fcf2faf20393add9
• c4762e2ebf41d1be601bd43334813849401061de
• c6a4bc3476a3c27b1b9b87033734a29def997038
• c7126f4e7f32795d7949e048192b7a5b032980d9
• c95f29ff32b2c7989300a53456e36e63a514fdcd
• cadc642524fa3ef3003ee6709e35746a3e7e6168
• cb481efbd7dc57b71460684df381076ad4ca21b2
• cc936c684fadbd1245c337979105c3ee4ac01456
• ce4dbc1da418954535eab30339cd6c0257ca9d5d
• ce7aa2241ce03a621c33051ce97785c811b89a22
• cf5624653cd94e66d5d61c56c813d5253505fb48
• cf6bca6e0e04437315961e746479f8cbe966be55
• d43283ce900a7c5a7691a010592bdce0e51609bf
• d54b2bf8afcd23a4154fe8e140822a1ec80d819d
• d5d1164ea53e73973263419582755d7ee317d660
• d5d66c644b3f5586c304874ddbd4e37979e44f20
• d648d6f88bb0609e42e58dab7d9e1cbeae8fbf36
• d713abaa7db0f13d9dd9e34e7145f6120c129564
• d7c89ae168e6ea6f029f50d363719bcb268f7826
• d8f120b67c9de06e3627795cd8de3c8505e201db
• d92422eeb16c5142313e4ff11541b05f520d2168
• da12c0d800bd6bb27acb4ee7f54c9cdd5b5984dd
• da3ee8a99880354be2740be270368493bffa0f67
• develop
• df69a6b85a8e159821eadebf76035c56f7d989d2
• e08c65b7173f59fb89bda27ca3783cca24c6c021
• e115fab8f2cbb481ab58b56d837b3b65942d2fba
• e3ec608ddd0fd07978afc746aedcd37e74358a4a
• e4ae4510b0fa824652cbc0bdaf3969c3e44ab4ab
• e50f264978ec7d8eb7a6b104825a3cdac5f222ea
• e5238baf9af531258d1df0dafa6e832c54c67ccf
• e8fa8bef9a7324e8fb27cfaae47902553f741c99
• e9ccb08fa994468e761b265df8a9be24b44d0189
• ec2af84dd91552f648ac018d26635a6375161ef1
• ed2769d62f6b4f5ecbae01440d0512d23cefa49c
• ee2382cb86b795b02233536bd1e6ad2449421c38
• eeddb1b6776e57eb85db9b0461c2bd191893d6cf
• f07f4c8cd962e39d1fc14aceeb7f1e60f46fd623
• f27368cdb0380954c2effca3550be6d110f454a6
• f2c32b57778fbb21b74e9535443e2648e117f8cc
• f37bbc980f05d31d698434b166ccf6df03dcf477
• fb10bf6dba5c4b5684080be0c245911c23e1d9f3
• feat/new-monitoring
• feature/ben_unit_tests
• feature/bumps-r35
• feature/fs2-no-pulls
• feature/independent-loader
• feature/jsonpaths
• feature/loader-sqs
• feature/metrics
• feature/pool
• feature/sentry
• feature/shredder_sqs
• feature/stream-shredder
• fix-redshift-repo
• fix/238
• fix/enum-resize-problem
• fix/full_type_name
• ignore-all-enc-fail
• master
• refs/tags/0.6.1-rc6
• refs/tags/3.5.0-rc1
• refs/tags/5.4.0
• refs/tags/5.4.1
• refs/tags/5.4.1-rc1
• refs/tags/5.4.1-rc2
• refs/tags/5.4.1-rc3
• refs/tags/5.4.2
• refs/tags/5.4.2-rc1
• refs/tags/5.4.2-rc2
• refs/tags/5.4.3
• refs/tags/5.4.3-rc1
• refs/tags/5.4.4-rc1
• refs/tags/5.4.4-rc12
• refs/tags/5.4.4-rc2
• refs/tags/5.4.4-rc5
• refs/tags/5.4.4-rc6
• refs/tags/5.4.4-rc7
• refs/tags/5.4.4-rc8
• refs/tags/5.5.0
• refs/tags/5.5.0-rc1
• refs/tags/5.5.0-rc10
• refs/tags/5.5.0-rc11
• refs/tags/5.5.0-rc12
• refs/tags/5.5.0-rc13
• refs/tags/5.5.0-rc3
• refs/tags/5.5.0-rc4
• refs/tags/5.5.0-rc5
• refs/tags/5.5.0-rc6
• refs/tags/5.5.0-rc7
• refs/tags/5.5.0-rc8
• refs/tags/5.5.0-rc9
• refs/tags/5.6.0
• refs/tags/5.6.0-rc1
• refs/tags/5.6.1
• refs/tags/5.6.1-rc1
• refs/tags/5.6.1-rc2
• refs/tags/5.6.1-rc3
• refs/tags/5.6.2
• refs/tags/5.6.2-rc1
• refs/tags/5.6.2-rc2
• refs/tags/5.6.3
• refs/tags/5.6.3-rc1
• refs/tags/5.6.3-rc2
• refs/tags/5.6.4-rc1
• refs/tags/5.7.0
• refs/tags/5.7.0-rc1
• refs/tags/5.7.0-rc2
• refs/tags/5.7.0-rc3
• refs/tags/5.7.0-rc4
• refs/tags/5.7.0-rc5
• refs/tags/5.7.0-rc6
• refs/tags/5.7.0-rc7
• refs/tags/5.7.1
• refs/tags/5.7.1-rc1
• refs/tags/5.7.1-rc2
• refs/tags/5.7.1-rc3
• refs/tags/5.7.2
• refs/tags/5.7.2-rc1
• refs/tags/5.7.3
• refs/tags/5.7.3-rc1
• refs/tags/5.7.4
• refs/tags/5.7.4-rc1
• refs/tags/5.7.5
• refs/tags/5.7.5-rc1
• refs/tags/5.7.5-rc2
• refs/tags/5.7.5-rc3
• refs/tags/5.7.5-test1
• refs/tags/5.7.5-test2
• refs/tags/5.7.6-rc1
• refs/tags/5.8.0
• refs/tags/5.8.0-rc1
• refs/tags/5.8.0-rc11
• refs/tags/5.8.0-rc2
• refs/tags/5.8.0-rc3
• refs/tags/5.8.0-rc4
• refs/tags/5.8.0-rc5
• refs/tags/5.8.0-rc6
• refs/tags/5.8.1
• refs/tags/5.8.1-rc1
• refs/tags/5.8.2
• refs/tags/5.8.2-rc1
• refs/tags/6.0.0
• refs/tags/6.0.0-rc1
• refs/tags/6.0.1
• refs/tags/6.0.1-rc1
• refs/tags/6.0.1-rc3
• refs/tags/6.0.1-rc4
• refs/tags/6.0.1-rc5
• refs/tags/6.0.1-rc6
• refs/tags/6.0.2
• refs/tags/6.0.2-rc1
• refs/tags/6.0.2-rc2
• refs/tags/6.0.2-rc3
• refs/tags/6.0.2-rc4
• refs/tags/6.1.0
• refs/tags/6.1.0-rc1
• refs/tags/6.1.0-rc2
• refs/tags/6.1.0-rc3
• refs/tags/6.1.0-rc4
• refs/tags/6.1.0-rc5
• refs/tags/6.1.1
• refs/tags/6.1.1-rc1
• refs/tags/6.1.1-rc2
• refs/tags/6.1.1-rc3
• refs/tags/6.1.1-rc4
• refs/tags/6.1.1-rc5
• refs/tags/6.1.1-rc7
• refs/tags/6.1.2
• refs/tags/6.1.2-rc1
• refs/tags/6.1.2-rc2
• refs/tags/6.1.2-rc3
• refs/tags/6.1.3
• refs/tags/6.1.3-rc1
• refs/tags/6.1.3-rc2
• refs/tags/6.1.3-rc3
• refs/tags/6.1.4
• refs/tags/6.1.4-rc1
• refs/tags/6.1.5
• refs/tags/6.1.5-rc1
• release/0.18.2
• release/1.0.0
• release/1.0.1
• release/1.1.0
• release/1.1.1
• release/1.1.2
• release/1.2.0
• release/6.1.3
• release/6.1.4
• release/6.1.5
• release/r33
• release/r34
• release/r35
• snyk-vulns
• use_semaphore

Build # 11181883606

Build Type

Pull #1363

github

Committed by spenes

Commit Message

Fix NPEs in TypesAccumulator

We've seen exceptions in spark executors like:

```
java.lang.NullPointerException: Cannot invoke "scala.collection.mutable.Set.isEmpty()" because the return value of "com.snowplowanalytics.snowplow.rdbloader.transformer.batch.spark.TypesAccumulator.accum()" is null
```

The error is coming from our Spark Accumulator for accumulating Iglu
types. This is similar to [an issue previously seen][1] in Spark's own
`CollectionAccumulator`. That issue [was fixed in Spark][2] by making
the accumulator's internal state non-final, and synchronizing access to
the internal state. So here we make the exact same change to our own
Accumulator.

It is a rare race condition which is hard to reproduce.

[1]: https://issues.apache.org/jira/browse/SPARK-20977
[2]: https://github.com/apache/spark/pull/31540

Pull Request Pull Request #1363: Fix NPEs in TypesAccumulator

Run Details

13 of 14 new or added lines in 1 file covered. (92.86%)

11 existing lines in 9 files now uncovered.

2321 of 4463 relevant lines covered (52.01%)

0.83 hits per line