Bodies started dropping; packets clashed like tumbling dice. A scale-up of a sensor monitoring network became unstable when agent processes began losing network connection for many seconds. Should have close to zero latency except  for reasons yet to be known, traffic congestion is affecting a (small) mesh of sensors. Sensors are great when they just work. Like stool pigeons, what they give you must be weighed against the right backdrop and confirmed with other source. I've had sensors too close to CPUs where the heat bollixed the measurements, and lately I've looked at one that seems to report abruptly different values, like a liar that changes their story every time it's repeated. A worse scenario, the subject matter of this case, is not just one bad apple, but a flock of remote devices that are going in and out of view. Because this is an open (unsolved case, tip line is open) issue, I'm posting as an article rather than a blog post, which I intend to write up if/when the mystery is solved. (Also, I can post the article under Open Source topic directly, where posts need approval). The open source under discussion is Zabbix, and I've "downsized" what I would do on a global network to a lab (home) scale. Beneath the covers is an open source Postgresql database, the Zabbix server (kernel) level, and what I'll show are frontend web console views (the GUI). Last 1 Hour data chart One hour view of an agent shows drop-outs over 50%! Yah, what is causing this? Put on the detective cap and plan the investigation. First, look back in the archives for priors. 1 year. One year agent up/down chart. There are definitely significant time periods of both 100% and "way off" between 30 and 60%. I like the click-drag to zoom gadget the Zabbix frontend allows (though not on Android tablets) a pinch to zoom. So the next step is to focus on when the bad news began. Zoom into a few weeks of data from 1 year level. Now what? Check the logs for any installs or tweaks at the first "tickle". If that is unfruitful, get out the Persian slipper and ponder a root cause analysis: network congestion (self-looping ddos?) network driver faults data reduction / prevention capacity plan mismanagement wrong tool for the task wi-fi / wired or other data collection protocols minimum configuration to duplicate I did the minimum isolation technique with an BSD-licensed NetBSD 10 server on a 2-core AMD64 machine, added one agent, and shortly duplicated the symptoms (or re-observed) per the server logs: Log snips: 4857:20240615:033829.178 resuming Zabbix agent checks on host "b1": connection restored 20267:20240615:033911.080 Zabbix agent item "system.swap.size[,free]" on host "b1" failed: first network error, wait for 15 seconds 4857:20240615:033928.859 resuming Zabbix agent checks on host "b1": connection restored 15939:20240615:034021.875 Zabbix agent item "vm.memory.size[available]" on host "b1" failed: first network error, wait for 15 seconds 4857:20240615:034026.795 enabling Zabbix agent checks on host "a1": interface became available 4857:20240615:034033.494 Zabbix agent item "vm.memory.size[total]" on host "a1" failed: first network error, wait for 15 seconds 4857:20240615:034038.584 resuming Zabbix agent checks on host "b1": connection restored <to be continued>  ... View more

Volunteering to contribute to Open Source projects may not be updating code, it might be running code,  observing results, and sharing conclusions. After participating in software beta testing over decades, I have both guidelines for running quality assurance tests, and a philosophy of searching for glitches in the matrix.   ... View more

The secret to accelerating your learning isn't what you think. It's about becoming a teacher...even when you feel like a student. Discover the 'learner-leader' model that's transforming professional development. #Dojo ... View more

As Open Source languages go, Python is one of them. Whether you use it directly or not, much Python code has leaked into the SAP ecosystem. I will describe challenges I have seen with setting up and maintaining various Python versions on UNIX-like systems. After I started writing this, yet another software compromise was published, and I will touch lightly on risk management. ... View more

A few weeks ago I related a work-flow tale that had a partial germ in a stack of (blank) punch cards from early IT days with the title "information archeology". Then a week ago an open source integrity issue appeared in the news as an intrusion in a common archive protocol repository. Meanwhile, as I continued digging through an accumulation of magnetic media I noticed a backup protocol I had not seen before. Amidst other standard "DOS format" floppies from the 1990s, a set like the one pictured here were hand-labelled with the contents, and the storage method, cryptically written as "fpio -0.". The author used GNU tar, on an AT&T UNIX system, but instead of tar used in the literal sense of a tape archive, The first line says "3b1/source (gnu)" which looked a bit like a Usenet group as the provenance for the origin of the code. Except the paper listing indicates what was stored were primarily compiled binaries, not source text (not the "*" below indicating executable). In the top level directory, there is a UNIX compressed (not GNU gz'd) archive file called "gzip-bin.tar.Z" and an executable named gnutar. The only other file at the root is "0.index" which is what the printout shows (highly probably). In the ./bin directory are compress/uncompress utilities from  the open source GNU project. Hopefully you will guess why there is gzip and gunzip, but only uncompress, not compress. There may be no files in either info/ or man/.  My earliest recollection of "production" backup to tape protocols was whether one would use the (a) tape archive tools, or (b) the cpio alternative, with the difference attributable to "this one comes from Bell Labs in New Jersey" and "this one comes from Berkeley, California." I had not heard the method of treating a floppy disk like a single-stream write/read cycle, like the reel-to-reel mag tapes you see in 60's spy movies, because the commonly adopted schemes were random or direct I/O instead. "cpio" standing for Copy In Out, my favorite set of options had the mnemonic "poodle move" but was spelled "-pdlmuv" that worked best. Could we read an fpio-formatted disk these days? Stay tuned...   ... View more

As I browse online analyses of a compromised compression algorithm library, my mind drifted back to the early days of free UNIX and open source software, particularly the "birth" of GNU Z, the LZW algorithm, and the refreshing alternative "zip." My first UNIX experience was a Unisys AT&T System VR2, then later VR3, which among its other built-in software had the compression program aptly named "compress" that tool typically text files and shrank them to a tenth of the usual storage space. You may still (or "again" depending on your timeline) find this inside a range of operating systems. Like this:   [...] EXIT STATUS The compress utility exits 0 on success, and >0 if an error occurs. SEE ALSO zcat(1) Welch, Terry A., "A Technique for High Performance Data Compression", IEEE Computer, 17:6, pp. 8-19, June, 1984. HISTORY The compress command appeared in 4.3BSD. NetBSD 10.0 January 23, 2003 NetBSD 10.0   Terry Welch had the "W" in the LZW protocol and worked at Sperry (one of the former companies that became Unisys). Because of law-stuff you can research, the distribution of this handy utility became problematic, so the GNU open source developers created an alternative tool that could (a) uncompress ".Z" files, (b) ran faster, (c) compressed more, and (d) was copy-lefted. Named gzip instead of compress, it liberated us from law-stuff, mostly.   [...] EXIT STATUS The gzip utility exits 0 on success, 1 on errors, and 2 if a warning occurs. SIGNALS gzip responds to the following signals: SIGINFO Report progress to standard error. SEE ALSO bzip2(1), compress(1), xz(1), fts(3), zlib(3) HISTORY The gzip program was originally written by Jean-loup Gailly, licensed under the GNU Public Licence. Matthew R. Green wrote a simple front end for NetBSD 1.3 distribution media, based on the freely re-distributable zlib library. It was enhanced to be mostly feature-compatible with the original GNU gzip program for NetBSD 2.0. This manual documents NetBSD gzip version 20170803. AUTHORS This implementation of gzip was written by Matthew R. Green   This man page mentions "zlib", for reference, also a handy base library for compression utilities.   SEE ALSO RFC 1950 ZLIB Compressed Data Format Specification. RFC 1951 DEFLATE Compressed Data Format Specification. RFC 1952 GZIP File Format Specification. zlib: http://www.gzip.org/zlib/   Once system administrators got wind of "gz" versus "Z", some of us went with the former, others who needed to report to stricter management could not. I remember hacking away at a Prime system that we got to supplant the Unisys UNIX host, with a bit of BSD and a bit of AT&T. Getting the GCC compiler to work was needed first, usually, as the stock K&R tools were not sufficient. Fixing compile time errors by removing or editing code got me the hairy eyeball from a peer, but I got gzip built and on the plan. Among the people who stood out (see, e.g. a writeup on the RFC 1950: https://datatracker.ietf.org/doc/html/rfc1950) were Jean-Loup Gailly and Mark Adler, the latter tagged as "original Zip author; UnZip decompression". One of the cool facts was the overlap of space exploration with algorithm development. Mark Adler had a day job at NASA "driving Rovers" and was hence, well positioned to leverage that research with compression code to get pictures from Mars as fast as the speed of light would allow. I'll also mention Greg Roelofs (see: https://infozip.sourceforge.net/) who helped me when I attempted to fix something in the UNIX X legacy program xpaint. You will likely see the results of that experience when he reviewed the XPaint code base and found a terrible design flaw that mangled color palettes when editing images. You may still see that warning today (as I do):   XPaint uses the native display format for storing image info while editing; the original image information is thrown away. This means that, in general, color information is irretrievably lost when using any display depth less than 24 bits.   So I like to say I didn't create this bug, or squash it, I just was lucky to work with someone who spotted the design flaw when I asked about another issue. EYE CHART:   - - - - | Z | | gz | | ZIP | |Info-Zip| - - - - -     ... View more

Discover the transformative journey of software development at SAP. Understand the role of InnerSource principles in fostering collaboration, learning, and community building, highlighted by the importance of the "First Contribution" and patterns like "30-Day Warranty" and "Praise Participants". ... View more

A short demo on how to translate a REST service ( running on Quarkus ) into a ODATA V4 service based on SAP CAP.  In the end all services can be deployed on KYMA for end to end testing. ... View more

Building a RESTful application with Quarkus on KYMA which sends message to Kafka using smallrye and reactive messaging. ... View more

This is part 4 of 5 TRM related blog articles: Part 1: Introduction Part 2: Install your first package Part 3: Publish your first package Part 4: Dependencies Part 5: Setup a CI/CD pipeline with Github Actions  What's a dependency? A dependency is a term used to describe the usage of an external package. Here's an example in the ABAP world: In this example, I created a new package called ZTRM_DEPENDANT and, as you can see, it contains a simple program with a write statement. The problem with this package is that, by itself, it won't work because it's using a method from a class in another package (ZCL_TRM_DEPENDENCY). So if I want to share the source code of this report, I also have to share the source code of the class used, otherwise you'd get a syntax error. The same goes if I have to transport this package to another SAP system. Can you imagine what happens when you have MANY dependencies? I'm sure you too spent some time in STMS logs finding the correct order of packages to transport... To solve this problem, developers declare a list of dependencies - packages needed (beforehand) in order to guarantee the software works as expected. Declaring package dependencies with TRM As explained in the earlier articles, TRM introduces manifests (package descriptors) in ABAP. Although you can always make manual changes, for some objects, dependency recognition is done automatically. I'll try to publish this package, but notice that TRM will not let me continue. In order to declare a dependency, you first have to publish that dependency. To fix the error, I'll first publish the package that contains the class ZCL_TRM_DEPENDENCY, and I'll name this TRM package trm-dependency, version 1.0.0. Now that the class ZCL_TRM_DEPENDENCY is contained in a TRM package, the dependency will be detected, and I can finally publish my package, trm-dependant. Installing a package with dependencies When you install a package with dependencies, TRM will first check if a release within the declared version range exists in the registry (or is already installed in the system) and will try to install that before any other operation. With TRM setup on your system, you can try installing this demo package yourself: run the command   trm install trm-dependant   you should see a message that explains what dependencies are needed One of the problems that may occur is circular dependencies (which means package A has a dependency with package B, and package B also has a dependency with package A): these situations should be avoided, but steps can always be skipped manually in order to fix this issue. Another scenario is two distinct packages with the same dependency but different version ranges: this is not an easy topic, and it's best discussed in the official documentation. Understanding dependency version costraints In the example above, the dependency with trm-dependency is automatically detected and set to require the package with a release in the constraint ^1.0.0. TRM uses Semantic Versioning, and this constraint means that upon installing trm-dependant, the system needs trm-dependency with a version between 1.0.0 and 2.0.0 (excluded). The reason why "^1.0.0" was chosen during the automatic detection is because TRM assumes that every non major release after 1.0.0 (the version installed on the system during publish of the dependant) doesn't have any breaking changes but rather fixes and minor features. This is to ensure that whenever you install trm-dependant version 1.0.0 you'll have the expected functionalities during its release, but you'll also have the latest non breaking changes version of trm-dependency. It's worth mentioning that dependency version ranges can be changed manually. Verifying dependencies Installing dependencies can be a scary task. In a situation where there are multiple packages required, you may not have the time to check every single one of them and their content. Checksums are introduced to basically ensure that the content of the dependency during the publish is the same during the install. Once a package is published, its checksum cannot be changed. This allows you to install in "safe mode" (perhaps installing open source projects): TRM will install the minimum version of the dependency within the range provided that will match the checksum declared during publish. This, however, is not done by default, as the expected behavior during the installation of dependencies is getting the best match within the declared range, as explained earlier. Dependencies with SAP packages But, are dependency declarations limited to other custom packages? No, they are not! Every time you use an SAP standard object, you are essentially creating a dependency. Because of how complex SAP standard objects are delivered, you can't just assume all the standard objects used by a third-party custom package are available on your target system. These dependencies are called, in TRM, SAP entries: Similar to dependencies, they can be automatically detected, but the only check executed is their availability on the system. Let's make an example:    Here I'm using a method from the class /UI2/CL_JSON. This is a standard class, and if we dig into its documentation, we'll find out that it's delivered with the UI2 Add-on for SAP_BASIS 740 – 76X (lucky for us, this has clear documentation, but it's not always this easy 😅). Unfortunately, semantic version is not something in use with SAP standard packages, so all we know is that the UI2 Add-on is required in order to use this class, but setting a range for it would be a challenge. After understanding what dependencies are, we can say that: If I were to copy and paste the source code of this program on a system without the UI2 Add-on, I would get a syntax error because I'm missing the required class. Even if the add-on is installed, we can't guarantee the expected behavior because the versioning used by add-ons is not compatible with semantic versioning. TRM solves point 1 by adding the class /UI2/CL_JSON to the required SAP entries: the expected manifest is   ... "sapEntries": { "TADIR": [ { "PGMID": "R3TR", "OBJECT": "CLAS", "OBJ_NAME": "/UI2/CL_JSON" } ] } ...   so during install, a check on the existence of the class is performed. It's not enough, as explained in point 2: the example uses the constant /ui2/cl_json=>pretty_mode-camel_case which was introduced in one of the rather newer versions of the add-on. To fix point 2, you would need to manually fine-tune the JSON in order to execute deeper checks during install of the package.     Next blog post: Setting up CI/CD workflows with Github Actions (Part 5). If you want to know more about TRM, check out the official documentation. ... View more