So, behind the scenes, a great Python has been skipping pebbles at my window.

Match Group, LLC has sent an unsuccessful DMCA, which I promptly destroyed because it was ridiculous.

I spoke to their legal team by request, and we came to an agreement.

Then, tonight, out of the blue, I got another one, this one is even better.

I develop a product called Machapi, formerly called Pofapi, and the name was changed by their request to avoid violating their intellectual property.

And, once again they don’t have a case and they’re trying to bluff their way into intimidating me into not developing the product.

These guys do not even know what my Kung Fu looks like.

As a result of provocation and legal threats, I will be developing Machapi into a fully featured, open sourced, polished engine for for accessing all of the subsidiaries and services where allowable by law.  What is not known is that I’ve already built out most of this and have the calls already documented for the extraction paths, so, I’ll be literally building from memory.

In a way, I kind of needed this, as I was feeling uninspired lately while I caught my breath.

For those of you who know my alter-ego, Bagira, I’m sure you know how this will play out.

Phase 1:

Phase 2:

–Bagira

Foster Git Repo Given a Second Chance

Some time last year I accidentally pushed some source tarballs that created huge objects in the git repo for Foster.

Our newest contributor repaired it but we lost revision history in the process.  Since we’re still in pre-release and working out design still I’ve gone ahead and approved – we won’t miss those early pushes of just trying things to see if they work for some of the early R&D.

I’ve archived the original with the bloated repo objects in its revision history as those tend to be rather important later sometimes, but, we really don’t need them.

Foster is now Makefile driven and Completely Dockerized

Wrapping up the process in a docker container was pretty important for this project as we needed the build host to be abstracted away from the user’s operating system without requiring much setup from the user.

Using the dev’s host as a build host I don’t think is very safe because of all the things that can go wrong.  This was exemplified by a destroyed host after a typo in a log cleanup script last year.

We made it safe by wrapping the examplar build up in a docker container and image build process and orchestrating all of that with a parent makefile.

The usage details are in the Foster readme.

The only prior setup or dependencies Foster has now on the build host is the presence of docker and make on the system.  It should compile on Windows, Linux, or Apple operating systems now as a pleasant side effect.

We have a simple roadmap established for Foster

Foster will use the Fedora base docker image as its build host.  I’ve got an updated snapshot of Fedora so that folks don’t have to wait on crazy download times.  I’ve also implemented source caching for the libraries it builds into the chroot.  This will still require users to download it once by design.

Once the first bootable Foster ISO is created we’ll dockerize that image and use it as the base, and then go back and edit Foster to run on itself to remove our dependency on the Fedora base image.

This post is intended to provide a rundown of exactly what we’re doing and why, to comprehensively include all of our updates and ambitions as they currently are, as there have been some direction changes that are notably significant, and, it seemed like it would be good to create a view for people to see if they feel like jumping in to help out.

This project has several goals and products associated with it, all related and towards a single end.

 

SURRO Linux:  A Basic Product

On the outside, we’re building a new Linux distribution — a very clean one intended to appeal to engineering and scientific environments.

The distribution is called SURRO Linux.

 

It’s Open-Standards Centric

During a time when critical standards that made Linux the engineering marvel that it is are no longer being adhered to by all major distributions, SURRO Linux will ultimately be LSB 5 certified and FHS 3.0 compliant unless a newer version of either of these standards is out before our first release.

There will never be a question about where a file should exist on the system.

Binary compatibility will be limited only by the LSB 5.0 standard.

 

It’s Not Based On Another Distro

SURRO Linux will not be based on another Linux distribution and will not be a child distribution.

 

It’s Raw and Fresh

There will be no copying of precompiled binaries from an ancestor distribution for its creation.  The entire distribution is compiled from raw source code for all components before being distributed as a binary package centered distribution.

 

It’s not just a Distro, Though

Along the way there are several tools that accomplish the creation of the distro for us that we’re building to get there.  These tools are designed to be either generic in purpose or to help others develop better products.

Aside from the technical merits:  SURRO Linux is not controlled by corporations, or investor interests — but technical needs.

 

Build Your Own Distribution: Foster

We’ve brought the Infrastructure as Code (IaC) concept to the OS Layer in the OSI model.

Foster is an intermediary distribution that can be easily modified to create your own Linux distribution.  We use an internally created build system that serves as configuration for the generated product, which means that the entire build process for the whole distribution, from sources, is automated and version controlled.

This will mean that the problems that we’ll be solving to get a bootable environment from nothing but source will be shared with the community, allowing others who might do better or have more resources to do so in a version-controlled, auditable manner.

 

We’re also targeting Container Crowds

This is also important to cloud and container solution providers in creating very minimal or controlled images with a great deal of granularity about what gets put on the image, auditability, change control and more for their base images — no more golden images that get cluttered over time — a version controlled image configuration is the apex of controlled container image management.

While we will be using Foster to create the SURRO Linux installer ISO, you could build your own — ISO or docker image.

 

It’s Host OS Agnostic

We use a docker container to wrap the build process for host/build OS abstraction.  Once the First bootable Foster image is reached we will be using it as our source image for that container and rewriting Foster to compile on itself.  This is for the purpose of providing the clean image to the community and to remove dependencies in our automations on third party providers.

The end result is that Foster should be able to compile on Windows, BSD, Linux, or even OSX in a very clean way.

 

Build and Automate Anything: Examplar

Examplar is an automation orchestration and rudimentary configuration management tool, or a build system, depending on how you look at it.  During the process of compiling Foster, Examplar is the tool that gets executed to kick off and orchestrate all of the build scripts and automations in place.  It is configured entirely in JSON allowing levels of complexity able to support automation of the scale required to build an entire operating system from source.

Examplar is its own product, generic in purpose, written entirely in C++ with no external system library dependencies, meaning it can compile and run just about anywhere on just about any operating system.

Examplar is also evolving as we build Foster with it, with new features being added frequently to accomplish our goals.

 

SURRO Linux

Once Foster is released we will be using Foster as the base image for the SURRO Linux installer ISO — we will be using Examplar to drive the configuration and installation of a new SURRO Linux system.

The first release of SURRO Linux will use RPM as a package manager, systemd as its init system, and will have public binary package repositories.  It will be curses-driven for installation.

Later releases will have menu options to select preferred components, such as bootloader, potentially the init system, package manager, desktop environment, and more.  Later versions are also intended to accept “profiles” that select these options in an unattended manner for CI systems.

The project is slow-moving, mostly due to resources being needed and research time.  With an increased headcount we could move faster.  With funding we could move faster.  The SURRO Linux project is currently entirely funded by Chris Punches in conjunction with donations to the project GoFundMe.  Join or contribute today.

I’m not dead, really!

Long break after some contract work.

I have found a way to successfully abstract away the host’s build environment, allowing us to move a little faster.

Rather than be informative and posting the link to the updates, I’m going to leave this announcement as is, mostly to give me time to integrate everything into the new build format and make all those minor decisions needed to get it all to work together.

This is going to be awesome.  If this works as well as it seems to so far, when it comes time to build Foster, you could even do it on a windows system with no difference in result.  Not that we’re going to try for that, but, it is expected to be possible.

-C

I’m semi-back.

Man, I don’t even know where to start.  So much has happened.

I’m really wiped out from moving.

Car: ’04 Saturn Ion II -> ’15 Dodge Challener SXT+

Major upgrade to a 2015 Dodge Challenger SXT Plus.  I’ve wanted a challenger since I was a kid.  It’s got a lot of horses and they’re all angry.  This thing is my chariot.  I, of course, bought a nice leather jacket and some ray bans to match it.  It’s setting me back for sure.

Career

Uhhh oh, I decided to take an FTE role a few months into my contract to create some work/life balance for my personal projects again.

That’s got me starting a week from today.  Unfortunately I’d wrapped up everything I need to do at that contract, so, knowledge transfer complete, the proper salute, and I’m on my way.  That’s got me going almost a full month without income.  Goodbye, savings.  Again.

Apartment

My lease ran up.  I moved to Dublin.  I wish I had a couple years ago, this part of town is way better than I thought it would be, and this apartment comes with a free gym/spa membership with unlimited access to hot tub, steam room, sauna, various pools, and exercise resources.

I’m actually still moving.  I’m taking my week gap between roles to move slowly so I don’t throw out my back.  I’m too old to be lugging shit up and down stairs.

SURRO and SILO

I’m getting the gears moving slowly again. Most of my research efforts revolve around containerizing everything sustainably.

SURRO Linux

I’m researching using docker to assist the build process so we’re not so hung up on build host architecture mismatches.  This would also eliminate the need to be root on the build host for many of the steps we’re looking at.  So far it actually seems viable.

IRCThulu

I’m going a different direction with it.  Also looking at containerization but we may be using some more powerful tools in the mix.  This may involve a full redesign.

Orchard

No updates. I just need to start developing on it again.  I used a waterfall process with this one so it’s literally just waiting to be implemented.  No real issues so far besides getting time, which should be better as time goes on now with all the changes.

CI/CD

I’ve gone ahead and created a CI/CD pipeline for all web resources in SURRO and split everything into a PROD and PREPROD.  We don’t need anything more robust than that until we have users.

I’m taking a break for a few days.

I’ve been working non-stop on planning and coding and planning for coding and planning for planning for coding for like a month.

I’ve got a new contract starting Wednesday, so, I kinda need to decompress before those start to manage the stress of learning large environments during the short durations these contracts require.

I’ll probably still end up coding because I always do that when I say I’m taking a break, but, time to sit back and play video games and eat junk food for a few days.

The pistache documentation site is down (again), so, I won’t be working on orchard for a bit until it’s back up.  This should give me an opportunity to:

Fix the new Antikythera theme:

• As a user I’d like a layered transparency background option with multiple images as a new feature.
• As a content author I’d like the preformatted text to be contained in the “paper” dimensions with a horizontal scrollbar.
• As a content author I’d like the page views to have the same background options as a post view.
• As a content author I’d like the block indentations to be visible again.
• As a content author I’d like the bullet lists to appear again.

Do some work on Foster:

• Fix the death-bug.
• Fix the cross-compiler issue and get a chrootable environment again.
• Create a better SDLC

Clean up content on what is now various blogs to make it easier to track how these projects all fit together.

After considerable thought I’ve decided to release IRCTHULU entirely as an open source product under a LGPL license.

I had held off on this initially due to concerns it would be abused, but, after working with it for a while here and there I don’t think it will be a huge problem if it is.  This will also open it up to potential 3rd party contributors to potentially get a better product.

This will also serve as katas for operationallizing the distribution components as that is an area growing a little rust for me due to all the skillset atrophy I’ve acquired over the last year or so.  It’s going to take me two years to catch up to where I was two years ago. :/

So, I haven’t posted in a while.

No, I’m not dead.  All projects are still inching along.

I’m doing some work on a new system called orchard.  It’s almost ready to be given its own dedicated site, but, I’m still implementing some of the database layer.

You can read about it here:

T-ORCH SPEC

It was originally called T-ORCH but that wasn’t going to show up in google searches so it’ll be marketed as Orchard.

 

SURRO has hit a blocker.  There’s something going on with the cross-compiler and it’ll be a decent amount of work to untangle it and it’s become apparent I’m going to have to do all the work on this project until alpha release.

-C

Antikythera is a WordPress theme designed with simplicity and stability in mind.

It is based heavily on both Artifact for Hexo as well as Hitchcock for WordPress.

You can download it from its dedicated github repository here.

FakeLA is an ansible distribution ready for quick deployment of a secure installation of Filebeat, Apache, Kibana, and Elasticsearch for the Linux Operating system.

Feel free to download it from it its dedicated github repository here.

 

Pofapi is a somewhat comical API library for POF.com written in Python.  It allows all sorts of new features for Desktop Clients.

You can read the documentation here.

I’ve decided to go ahead and prop up the mailing list as a new communication path.

I’m considering removing the bug tracker as while it was really productive for a while we just don’t have enough people until we get to release status.  Not decided yet.

I’m going to be doing quick bursts on the Foster build.  It will try to be distro independent for the most part but I’m on a Fedora machine — so in some places stuff will be specific to Fedora for the builds — it’s encouraged for people to go behind me and make stuff distro independent so I can just focus on getting us to a bootable state.

Mailing list url:

https://groups.google.com/forum/#!forum/surro-linux-greybeards

 

We may be losing some folks on the core dev team due to work/life balance needs — understandable and regrettable — contribute where you can.  It’s going to be the wild west before Alpha release.  That’s the primary goal first.  We’ll make it pretty when we get there.  Slightly less planned than we’d hoped, but we’ve gotta move forward on this project.

-C

I haven’t touched it since my last post.

We’re currently on Milestone 2.

I’ve been working on a closed source project called T-ORCH, an application-layer orchestration solution designed for large scale enterprise environments, as an effort to fund the SURRO Linux project.

Once sold I hope to work on SURRO full time after we get to a certain point, but it’s a long way there.

This largely stems from the fact that I totally suck at fund raising.  We actually have a fundraiser page where you can donate towards surro linux development at:

https://www.gofundme.com/surro-linux-development-funds

While what we really need is more people, every bit helps keep the surro linux project going.

I am concerned about how involved the design and development time is on T-ORCH and how it’s pulling time away from SURRO, so, from here forward I’ll be devoting a 5/2 split of time between T-ORCH and SURRO.

The main delay there was the design and implementation of IRCTHULU.  IRCThulu is a project of intentionally crafted traditional SOA components so that as T-ORCH is implemented it will have integration concerns in mind to make T-ORCH as well built as possible — and to have IRCThulu be a showcase for doing just that.  With IRCThulu created it’s now time to build out T-ORCH and then integrate it into IRCThulu components.

I’ll be spending Saturdays and Sundays on SURRO and Mondays through Fridays on T-ORCH until T-Orch is in release status and IRCThulu is integrated into an instance of it.

Some very interesting people expressed interest in the SURRO Linux project.

Anyone curious on joining the team of greybeards or general inquiries feel free to reach out to me personally at punches.chris@gmail.com.

I don’t think it’s any secret any more that this is one of my projects.

-Chris Punches

T-ORCH is the nervous system for SkyNET.  We’re not kidding.  It controls services and microservices in very large environments.  It is designed to be scalable almost without limit.

You can learn more about it at its dedicated page below.

T-ORCH SPEC

Felt good to get this one knocked out:

https://presenta.silogroup.org

That project took like a year of design in between other projects.  I needed a distraction from surro linux for a bit while I wrapped my head around the complexity of bootstrapping compiler builds.

New Year’s Eve is tomorrow.  The lady and I are going to a fancy place to watch the ball drop.

Speaking of balls dropping, the gym visits have got my test levels up.  Kinda digging the new me.  Or is it the old me?

I can’t tell.

Oh, I got one of those Amazon Echo Dot virtual assistant things for Christmas.  I’ve got it going up in the house and it’s pretty cool how it changes the atmosphere in the house.  It can do a ton of things.

I really want to make it just sound an alarm if one of my servers goes down.

IRCThulu is today.

There’s going to be alot of fallout.

Expected Kline

The freenode network is mostly impacted by this.  These are not people who negotiate.  I’m expecting that.

First, the Freenode network is going to kline all of my resources from the network not realizing that there are 3rd parties running the runners who do the logging and that it’s not up to me which channels they log.

Second, this will be the demarc for active development for IRCThulu allowing some time for it to burn in with changes here and there.

SURRO

From there I’ll be able to focus solely on SURRO while enacting the much needed sanctions that IRCThulu provides.

We’ll need to get that mailing list set up.

GitLab has become too resource intensive.  The server just wants too many resources for what it’s actually doing.  I’ll be looking at transitioning our repos to a more efficient system.

As for IRC presence, SURRO will still have one, and I will still be on the freenode network, I’ll just be doing alot of host-hopping in between repeated klines.  It’s unfortunate that this level of upset needs to be created to get such serious issues addressed properly.

The answer is, ultimately, to turn the list of bans in #surro-linux into a kline entry list, and, this is a permanent new age of accountability in these communities.

IRCThulu is one of the world’s largest IRC log aggregation systems.

You can learn more about it at it’s dedicated page below.

How It Works

For a demonstration, there is an example UI being hosted currently here.

I’ve decided to give IRCTHULU a new dedicated home.

http://ircthulu.silogroup.org/2017/12/23/new-dedicated-site-for-ircthulu/

It’s painfully clear the types of controversies IRCThulu will cause, and I wanted to document my adventure there in its own dedicated space so as not to clutter my already cluttered blog.

This blog was intended as a scratch board or hub for my projects, so, I can certainly cover things I’m looking at with it, but, it would not be useful to have all content from all projects in one blog.

This is one of the reasons the surro linux blog was moved to linux.silogroup.org.

After a week or so of tweaking the backend to get it stable and initial buidout of the decoupled API that serves the aggregated content — it’s safe to say that IRCTHULU is production ready.

Not only that, after some research it’s also pretty clear that we’re not just talking about a log aggregator — it’s the world’s largest IRC Log Collection Engine currently.

One thing I’m running into is delay in getting the data from the database due to the size of the tables the logs are stored in.

I may want to set up some jobs that create cached tables periodically to speed that up, but, every piece of action you take to do things like that take away from the realtime nature of the data itself.  I’ll have to give it some thought.

This content is password protected. To view it please enter your password below:

Password:

I reworked the consumer in data and have titled it synapse.

Performance Notes

Based on previous tests and the current behaviour of ircthulu’s tenta and data backend, I think we will not even need to consider scaling until we’re at about the 2000 simultaneous channels mark.

It’s there if we need it.  I can cluster and autoscale it easily.  Once we do that I seriously doubt if we could not get almost every major IRC channel on every major network logging simultaneously without significant data costs.

Stability

The Tenta layer is decentralized and distributed.  The MQ is clustered.  The DB is clustered.  The synapse between them is rather butch in the multithreaded manner in which it is processing.  I’m overall pretty confident in its ability to store the content without intervention at this point and am hardening the way in which it is bundled for distribution.

API and UI is next

From here it’s the API and the UI to actually serve the content, and just making sure the feeds are still coming in.  Some are already active.

Participants

If you have a persistent connection and would like to run the tenta client as a systemd background service please reach out at punches.chris@gmail.com with details about the channel you would like to host.

 

I’m pleased to announce the Tenta client came together quite well, marking the completion (and activation) of the data backend for IRCTHULU.

For those of you who haven’t been following this project, this is how IRCTHULU works:

Tenta is a pair of services comprised of the runner client, tentacle and producer service, called nerve. They run in the background of the systems of the people who collect the data, and are installable as an RPM.  The “transporter service” called nerve pumps the messages captured by tentacle into a message queue I currently own.  Tentacle is configured by the runner and can point to any channel they wish.  I can make recommendations about what channel but I can not decide ultimately.

From there a multi-threaded service called synapse in the storage layer empties out the MQ, transforms, and pushes the messages into a database.  From there, an API has readonly access to the databases to serve information relevant to supply a user interface on a remote site.  The MQ/DB/API combination is a layer called Data.

The UI will be called Presenta.

So that’s Tenta, Data, Presenta.

What does performance look like?

The total throughput of the system was last measured at 30,000 messages per second, allowing very many channels to be logged.

What kind of Data does it record?

Here’s a data sample:

{"type": "pubmsg", "nick": "Mihaita", "channel": "#freenode", "message": "Phanes @robot?", "timestamp": "2017-12-14 03:27:36.762231", "host": "166.ip-54-37-156.eu", "ident": "~SirNeo"}
{"type": "pubmsg", "nick": "Phanes", "channel": "#freenode", "message": "SirNeo-, yeah its part of a project called ircthulu, its all google indexed eventually", "timestamp": "2017-12-14 03:28:17.214279", "host": "surro/founder/phanes", "ident": "~Phanes"}
{"type": "pubmsg", "nick": "SirNeo-", "channel": "#freenode", "message": "i will google it about latter", "timestamp": "2017-12-14 03:29:09.333669", "host": "166.ip-54-37-156.eu", "ident": "~SirNeo"}
{"type": "pubmsg", "nick": "Phanes", "channel": "#freenode", "message": "SirNeo-, its not public yet", "timestamp": "2017-12-14 03:29:16.679930", "host": "surro/founder/phanes", "ident": "~Phanes"}
{"type": "join", "nick": "[skyline__O__]", "channel": "#freenode", "message": "", "timestamp": "2017-12-14 03:29:22.442250", "host": "1.180.74.58", "ident": "~Shen"}
{"type": "join", "nick": "blkshp", "channel": "#freenode", "message": "", "timestamp": "2017-12-14 03:29:43.017512", "host": "about/windows/staff/blkshp", "ident": "~blkshp"}
{"type": "join", "nick": "SirNeo-", "channel": "#freenode", "message": "", "timestamp": "2017-12-14 03:29:49.518446", "host": "unaffiliated/sirneo", "ident": "~SirNeo"}
{"type": "pubmsg", "nick": "Phanes", "channel": "#freenode", "message": "SirNeo-, it uses a \"guerilla\" pattern for data collection points from disparate, distributed hosts via a background system process installed via rpm, so there's like no way to tell who it is and it doesn't log any events that could be used to detect the bot through the logs, also waits a random amount of time after starting to begin logging", "timestamp": "2017-12-14 03:31:17.932267", "host": "surro/founder/phanes", "ident": "~Phanes"}
{"type": "pubmsg", "nick": "SirNeo-", "channel": "#freenode", "message": "Phanes, how do you know we don't use allready a VPS?", "timestamp": "2017-12-14 03:31:20.409542", "host": "unaffiliated/sirneo", "ident": "~SirNeo"}
{"type": "pubmsg", "nick": "Phanes", "channel": "#freenode", "message": "channel redundancy", "timestamp": "2017-12-14 03:31:52.678278", "host": "surro/founder/phanes", "ident": "~Phanes"}
{"type": "pubmsg", "nick": "Phanes", "channel": "#freenode", "message": "autorecover", "timestamp": "2017-12-14 03:32:05.244805", "host": "surro/founder/phanes", "ident": "~Phanes"}
{"type": "pubmsg", "nick": "Phanes", "channel": "#freenode", "message": "bug tested for \"gotchas\" to kill the bots", "timestamp": "2017-12-14 03:32:21.725860", "host": "surro/founder/phanes", "ident": "~Phanes"}
{"type": "pubmsg", "nick": "Phanes", "channel": "#freenode", "message": "the metadata separation is the cool part though", "timestamp": "2017-12-14 03:32:53.327834", "host": "surro/founder/phanes", "ident": "~Phanes"}
{"type": "join", "nick": "skyline__O__", "channel": "#freenode", "message": "", "timestamp": "2017-12-14 03:33:03.015222", "host": "1.180.74.58", "ident": "~Shen"}


It’s recording the message type, although some message types are restricted (like some CTCPs or DCC) from logging, the channel the message occured on, the message itself if there is one, timestamp of the message (server), the host of the user submitting the message (or cloak if they have one), and the ident value associated with that host at the time of the message submission.  I will also tonight be adding in an even handler for nick changes to capture that metadata for nick changes.

That’s oddly specific.  What purpose do these datapoints serve?

Great question.  This will allow the UI to serve as a search engine of sorts, allowing you to browse logs, and, identify sock puppets, or trends, or even perform statistical research or machine learning algorithms upon for detecting anomalies.  I’ll explain one example use case:

A user FrankG likes to use sock puppets to harass people online.  He connects with FrankG!dirtyfrank@224.223.221.33 normally.  You suspect that he has goofed at some point and used the same ident on his irc client or host at some point breaking OPSEC.  This would easily be determined by simply looking at a log that captured frank talking by searching for username FrankG or going to a known channel and time you knew he was speaking.  From there you can click on his ident, dirtyfrank and see all the users captured who used that ident.  Or you can click on his host 224.223.221.33 and see all the users ever having connected and posted a message in a logged channel from that host.  Or you can click on his username, FrankG and see all the hosts and idents that have ever been used by the username and begin doing deepe analytics.  Obviously, the larger the dataset, the more effective it is.

This is in addition to a more important community need being filled:  IRC is a lost trove of knowledge for people looking to increase their knowledge in a technical area — these channels are like living textbooks for people solving problems, particularly on open source systems.  By providing this resource for as many channels as possible we are opening up those channels to google search indexing, allowing someone looking for a problem a greater likelihood of finding their answer when googling.  Cool amirite?

What does the legal model look like?

Great question.

I own a database cluster that stores the feeds and the UI, which are both on different domains and can be decoupled at any time and even given to someone else.  Both the database layer and the UI publish only the feeds that are given to it and all of the feeds that are given to it.  The feeds are public domain.

The runners are, of course, expected to comply with Terms of Use and EULA for any networks they connect to and I trust them not to create liabilities for themselves with it.  Ultimately this is a discussion between the runners and the network owners.

In terms of IP, I’m keeping it closed source, for now, but obviously if it serves my interests I can make it open source and release under AGPL.  My only concern is that if I do that it will be abused and create many spinoffs that I can not control.  And, obviously, if I have to transfer ownership of the DB or UI layers I’ll lose any ability to control content and will have to release the source.

What’s left to do before it goes public?

I’ve got some adjustments to make in the transformation layer of the synapse component to really polish it out and then it’s the API, UI, and SEO.

How do channels “opt out”?

Some channels or networks may desire not to be logged and this is certainly understandable.

They can simply email me at punches.chris@gmail.com and request it, stating why.

While a reasonable justification is required, the overwhelming majority of channels will be approved with no issue; some will be expected to make adjustments to qualify and some in particular should view those adjustments as sacrifices.

IRCThulu, ultimately, is a tentacle monster by design, it’s the size of a building, and it’s destroying reality on IRC.  Well, not all of reality, just a particular kind.

Cross-compilers are tricky business for this kind of operation.

At this point I’m still picking away at various issues surrounding binutils/gcc/glibc required to do the rest of the build out, but Foster is coming together.

Unfortunately, a community need that I’ve been putting off deserves immediate attention until it’s complete, and ircthulu is one of the components for that initiative, so, I’ll be devoting some time to that — and hopefully one of the greybeards will use that time to unkink the cross-compiler generation in Foster.

Ircthulu is such a large project that I don’t expect it to end soon and it’ll probably be a little controversial for a bit.

As mentioned previously, a new feature was needed to be added to Examplar to allow much greater simplicity in the Foster build to make its content more distributable and I could definitely see that affecting usage due to the removal of the need to hardcode paths in the various files.

So, we’ve added it.  Instructions are in the repo.

The source code for the 1.6b version is here.

The source code for Foster is here.

I believe we will probably have much slower updates from here.

Right off the bat I noticed that there are some features that will make the foster builds quite a bit easier.

The most prominent feature I’d like to see added next will be a config file that sets any enviornment variables the user would like any executables that get called to have in their environment.

Another is per-Task STDOUT/STDERR logging as it’s currently up to the user to implement this in their scripts.

Thoughts for future versions.  In our case we’ll need some variables set across several tasks or heals that will be executed, so, I’ll be sourcing a common file that sets these.

Another is a switch for setting the current working directory for execution context to prevent having to hardcode system-dependent paths all over the place in the units.  I will make sure that gets put in place next.

So you’ve got Examplar compiled and you’re ready to start automating the world.

If you’re thinking “how do I configure this thing”, this article is for you.

Definitions

Units

A Unit is an automation definition, written in JSON in a UNIT FILE.  Deeper into Examplar’s internals, Units and Tasks have slightly different functions, but for the purposes of users, the terms can be used interchangeably.  A Task is a task to be performed in a Plan, and a Unit is its definition.  A Unit is a JSON object that has:

• A name, which is an identifier for the Unit used by people.
• A target, which is the path to the automation script performing the work.  This provides a clean linear path for huge chains of scripts to be executed in order and tracked on return for additional logic in chaining.
• A rectifier, which is the path to the automation script to be executed if the target call fails.
• A rectify attribute, which tells Examplar whether or not to execute the rectifier in the case of failure when executing the target.
• An active attribute,which tells Examplar whether or not the Unit can be used in a Plan.  This gives Unit developers a way to tell Plan developers not to use the Unit.
• A required attribute which tells Examplar whether or not the Plan can continue if the Unit fails.  If the rectify attribute is set to true, this attribute is checked after a rectifier failure.  If not, this is checked after target failure.  In either case, if the rectifier or target do not return successfully, Examplar will halt the execution of the Plan if this is turned on for the unit being executed.  Otherwise it simply moves to the next Unit being executed.

Tasks

A Task is an action item in a Plan, just like in real life.  In the context of Examplar, a Task is a Unit that has been loaded and incorporated into a Plan in an actionable state.  Inactive Units can not be loaded into a Plan and thus can never be a Task.  The primary difference between a Task and a Unit is that a Unit is not actionable — it’s just a definition — while a Task a consumable, actionable automation.

Suite

A Suite is not visible to the user and this is only for informational purposes.  A Suite is a collection of all available Unit definitions loaded from one or more UNIT FILES.  Just as a Unit is the definition for a Task, a Suite is a collection of Units that define the Task components of a Plan.

A Suite is consumed by a Plan during the conversion of Units to Tasks, though this is not visible to the user — it just simply helps to understand the kind of abstraction taking place in the conceptual model of Examplar.

Plan

A Plan is the glue of all the components of Examplar and is deceptively simple.  A Plan loads a Suite for its Task definitions (Units), but the Tasks to actually execute are specified in the PLAN FILE.  The Tasks are executed in the order specified in the PLAN FILE.

FILES

CONFIG FILE

This is the one config file that Examplar uses.  The default path it looks is /etc/Examplar/config.json.

A config file at the time of writing this specifies a single JSON object with 3 attributes:

• The UNIT FILE path or a path to a directory containing unit files.
• The PLAN FILE path.
• The configuration VERSION.

UNIT FILE

The UNIT FILE is a specification of where the Units are defined.  All UNIT FILES in that directory will be amalgamated to generate the Suite.  These types of files must end in `*.units` for their filename.

PLAN FILE

The PLAN FILE is a specification of the order that Tasks are executed, and their dependencies upon each other.  Dependency implementation is a touchy matter that is pending implementation, so, mileage may vary until release.

Configuration VERSION

The configuration version is checked to ensure that the configuration is consumable by that version of Examplar.  This will pave the way for reverse compatibility if the project moves in that direction.

Man have I been busy since my last post.

I got the C++ rewrite of Tenta semi-functional and then hit a roadblock that would require a rewrite of the entire IRC event layer to really interface with NERVE the way I wanted it to.

So, naturally I shelved it because when I hit a roadblock like that staring at it longer won’t fix it.  Network programming sucks because of how optimized it needs to be, and I’m more of a prototyper.  The python version is still far superior but that’s just because the network layer is wrapped up very well in python.

In the meantime I brought Examplar not just to Alpha release status but through it and into Beta.  We’re on version 1.5b now and it’s looking really good.

Good enough to call it stable at this point, but, I want to hold off until it generates a bootable ISO on Milestone 2 for Foster (The SURRO Installer Image).  I have no doubt that as we get into the thralls of building a Linux distro that we’ll want to make some changes or optimizations that will make the first stable release that much better.

Been hitting the gym pretty good.

Putting Tenta on hold for a while to recover from the Examplar frenzy of the last week or so, and in the meantime I’ll pick away at some theme tweaking for the blogs and news feed.  Really need to get that linux blog at silogroup on this CNAME and this one moved over to silogroup though, the SILO/SURRO split was kind of half baked.

It’s been pretty peachy.  Getting my half-mile jog time down and am considering adopting a puppy if the landlord will allow it.

I’m unsure of what direction I want to take the new blog themes.  I really like having a bland appearance on the SURRO blog and absolutely must keep the “Phanes as Solaire screaming into a microphone over a nuclear cloud” theme.  Maybe I’ll just clean it up a bit.

I’m taking a break from projects for a few days and just having fun.  I don’t do that enough.

Examplar is a configuration management, automation, and testing tool with minimal runtime requirements, powered entirely by JSON and should integrate with almost any existing tools or frameworks.

You can read more about Examplar at its dedicated page below.  Examplar was built as a supporting component driving the creation of SURRO Linux:

Beta Release – Examplar 1.5b

SURRO Linux is now moving from milestone 1 to milestone 2.

In this phase we will build a bootable ISO where the rest of the build occurs for a lightweight environment from which SURRO will be installed.

Examplar will be the driving tool for these builds and configurations.

-CP

Examplar is now in Beta at version 1.5b.

I’ve removed all hardcoded paths, implemented all relevant commandline switches, usage docs, and more.

At this point the tool is to be used for all SURRO/Foster development.

This version by all manners looks and behaves stable, but, I’m not comfortable releasing as a stable version until it’s built a bootable ISO so that it can have that guarantee behind the release announcement.

Feel free to download it and use it for any unrelated projects.

I expect in a future version that we’ll want to have a switch to disable syslog logging for absolutely barren environments but I do not think that will be necessary at this time.

Since we’ve moved to beta, I will be offering public RPMs and spec files.

Most work from here related to development on Examplar itself will revolve around improving documentation on how to use it as this is a new tool in the Linux space so I expect alot of confusion at first.  It’s deceptively simple.

In the meantime the source can be downloaded here if you’d like to compile it yourself:

https://source.surroindustries.com/Cycle0/Examplar/repository/archive.tar.gz?ref=Examplar-1.5b

It is strongly recommended to package it before installing it due to the frequency with which we’ve been updating, but it really doesn’t need to be “installed”.

Here’s a script that will get you up and running. Make sure you have cmake installed on your system to be able to compile it. Cmake is a great tool for building C/C++ projects and more. You will also need a C++ compiler. I use G++.


mkdir ./Examplar
cd ./Examplar
wget https://source.surroindustries.com/Cycle0/Examplar/repository/archive.tar.gz?ref=Examplar-1.5b -O examplar-1.5b.tar.gz
tar xvf examplar-1.5b.tar.gz
cd Examplar-Examplar-1.5b-*
cmake .
make
./examplar -h


Thanks to all the wonderful people who have provided input on the direction of this project, and I look forward to what we do with this — and I know the SURRO Greybeards have been very patient while I tried to make the time to do this.  I can not say that making the time did not have consequences, but we made it and that’s what matters.

After a couple nights of grinding, we’ve got a major, major improvement from the v1.0a candidate.  Examplar v1.3a is released.

This version has major bug fixes and feature additions so it’s recommended to move to this version from 1.0a.

• This version repairs a rather serious logic bug in the task execution flow.
• There is also the new feature of now being able to split Units across multiple files in the same directory and pointing the config to the directory of .units files, which will give some more flexibility in unit definitions.
• It now logs to syslog.
• Reworked some of the exception layer to give more details to the user during failures.

I’m rather pleased with this version although I would like someone to come and clean up the code a little bit as there are a couple parts of it that are functional but ugly.

You can use whatever you’d like to monitor syslog — in my case I use systemd on my test machine, so I just do:

journalctl -t Examplar -f

And it will hold onto the syslog in realtime while I execute Examplar, giving me realtime logs.  It still dumps to STDOUT and STDERR. This should be relatively universal, but, like I said, use whatever you’d like to monitor syslog during Examplar runs.

At this point all original features are accounted for and it’s ready for some hands-on testing by building out Foster — but I think I’d like to hold this in Alpha before moving to a Beta or stable release.

-CP

We made it to alpha.  This is where the fun starts.

In Alpha status, we’ve got all the functionality of the tool implemented but will still be adding in some polishing as we encounter any issues using it to build Foster.  There will be minor features added while in Alpha status:

• Logging to file
• Loading a directory of UNIT FILES instead of just one UNIT FILE.
• Commandline switch processing.

Examplar is released under AGPL and requires only a C++ compiler and cmake to build, with zero dependencies.

Why Examplar Works

So, this has come up a bit, and I wanted to have something written up somewhere that explains how these parts all fit together.

The Solved Problem

During the build of the initial environment, there are alot of considerations and hurdles that come up depending on what distribution you are on, and what you want built or automated.

Needed a Boostrap With No Runtime

In our case, we’re going to be compiling into a chroot with basically nothing but a compiler, glibc, a kernel, libstdc++ and your fingers.  We needed a tool that could, in a distro-independent manner, start automating first the initial creation of that chroot environment, then do the compilations necessary to populate it with those things, and then still have enough runtime to bootstrap into itself to continue the rest of the setup of that environment in a way that accommodated more and more complex tools being executed which depend on runtimes provided by previous automated tasks that had been performed by the tool — in a chain of successive scripts that have no defined complexity limit.

We also needed a way to run environment checks and audits on one track of scripting and work to perform repairs when those checks failed, or alternative scripts to run when automations failed — and current automation tools that are distro independent require massive runtimes that just are not realistic to build by hand in a chroot (ansible, chef, puppet).

Coding Task diversity and Order for a Distro Build had Unmanageable Complexity

In addition to this, it became clear that the nature of automations needed for building these environments was so diverse that abstracting them into units for planning and quality checks was just a monstrous task and we needed a tool that forced us to abstract tasks into work units before planning what needed to be done.  This would also allow us to version control the plan for the work needing performed itself as opposed to managing that process through very talented humans — and we needed the plan to be decoupled from the tasks needing performed so that the abstraction necessary to create a distribution could be focused on without too much low level on implementation.  We also needed the tool to have some level of recursion possible.

We needed the Plan as well as the Coding Tasks manageable by many Contributors

I realized that most of the guides out there were for single users to build single user systems.  I needed a way to allow a large contributor audience to contribute not just to the tasks that were being coded but to the plan that defined the execution order, and what would be executed, in a way that allowed profiles in the build system.

We needed something with a Flexible Use case

We needed a tool that could be repurposed outside of our initial use case to create a toolchain that was reusable to save time since we have a small crew that could become large if we were to beat the odds and create a successful project.

We wanted to introduce a new tool of this type to the Linux world, and we wanted it to be dead simple.

Due to many of the reasons mentioned above it was important to me at least to provide a tool to the Linux world that did not have such heavy runtime dependencies and domain-specific knowledge required to use, while still being able to accomplish most of the net capabilities these tools provide.  We wanted to provide a tool for the systems-heavy DevOps admin and their development-heavy environments.

We wanted modern configuration style

And JSON was it.

That’s why work started on Examplar.

A Pivotal Moment: The Design Evolution

However, during the design process I realized that I could make this so generic that it could do everything without requiring a complex specialized toolchain as long as it was only providing “workflow” capabilities for already existing tools and resources on the system, allowing the automation to grow with system capability.

This would save the eye-crossing complexity of massive makefiles or cmake configurations, while providing a tool that could bootstrap into itself and continue whenever the work was moving to a new environment (a situation that will happen a few times during the builds from “your current distro” to “your booted SURRO Linux Environment”.

 

* * *

 

How SURRO’s Examplar Works

First, some definitions:

Units

A Unit is an automation definition, written in JSON in a UNIT FILE.  Deeper into Examplar’s internals, Units and Tasks have slightly different functions, but for the purposes of users, the terms can be used interchangeably.  A Task is a task to be performed in a Plan, and a Unit is its definition.  A Unit is a JSON object that has:

• A name, which is an identifier for the Unit used by people.
• A target, which is the path to the automation script performing the work.  This provides a clean linear path for huge chains of scripts to be executed in order and tracked on return for additional logic in chaining.
• A rectifier, which is the path to the automation script to be executed if the target call fails.
• A rectify attribute, which tells Examplar whether or not to execute the rectifier in the case of failure when executing the target.
• An active attribute,which tells Examplar whether or not the Unit can be used in a Plan.  This gives Unit developers a way to tell Plan developers not to use the Unit.
• A required attribute which tells Examplar whether or not the Plan can continue if the Unit fails.  If the rectify attribute is set to true, this attribute is checked after a rectifier failure.  If not, this is checked after target failure.  In either case, if the rectifier or target do not return successfully, Examplar will halt the execution of the Plan if this is turned on for the unit being executed.  Otherwise it simply moves to the next Unit being executed.

Task

A Task is an action item in a Plan, just like in real life.  In the context of Examplar, a Task is a Unit that has been loaded and incorporated into a Plan in an actionable state.  Inactive Units can not be loaded into a Plan and thus can never be a Task.  The primary difference between a Task and a Unit is that a Unit is not actionable — it’s just a definition — while a Task a consumable, actionable automation.

Suite

A Suite is not visible to the user and this is only for informational purposes.  A Suite is a collection of all available Unit definitions loaded from one or more UNIT FILES.  Just as a Unit is the definition for a Task, a Suite is a collection of Units that define the Task components of a Plan.

A Suite is consumed by a Plan during the conversion of Units to Tasks, though this is not visible to the user — it just simply helps to understand the kind of abstraction taking place in the conceptual model of Examplar.

Plan

A Plan is the glue of all the components of Examplar and is deceptively simple.  A Plan loads a Suite for its Task definitions (Units), but the Tasks to actually execute are specified in the PLAN FILE.  The Tasks are executed in the order specified in the PLAN FILE.

FILES

CONFIG FILE

A config file at the time of writing this specifies a single JSON object with 3 attributes:

• The UNIT FILE path.
• The PLAN FILE path.
• The configuration VERSION.

UNIT FILE

The UNIT FILE is a specification of where the Units are defined.  In future versions of Examplar this will actually be a directory and all UNIT FILES in that directory will be amalgamated to generate the Suite.  Currently this is only one file.

PLAN FILE

The PLAN FILE is a specification of the order that Tasks are executed, and their dependencies upon each other.  Dependency implementation is a touchy matter that is pending implementation, so, mileage may vary until release.

Configuration VERSION

The configuration version is checked to ensure that the configuration is consumable by that version of Examplar.  This will pave the way for reverse compatibility if the project moves in that direction.

 

* * *

 

What Examplar Looks Like

1. Write a script that does a thing, or checks a thing, or builds a thing, even calls a thing that does a thing.
2. Write a script that fixes conditions that you expect to prevent whatever you called previously from completing successfully.
3. Create a UNIT FILE defining those two scripts.
4. Create a PLAN FILE referencing the UNIT you created.
5. Point your configuration file at the UNIT FILE and PLAN FILE.
6. Run Examplar.  Eat Waffles if Necessary.

Experienced scripters will notice two things right off the bat.  For (2) you cant know all of the conditions, and if you could you can’t have a way to know which condition occurred.  This is accounted for in the design!

Write checks for each of your conditions as Unit targets.  Write heals for those conditions as Unit Rectifiers in their corresponding Unit.  Enable the Unit Rectify pattern.

Only you can predict the things that will go wrong, so you’ll want to approach it iteratively — which where previously was not possible is now gaurdrailed in a system that forced you to approach it as the creation of “code units” that “do a thing”, adding on as you encounter new ones needed.  This tool not only is possible to use in Agile environments — but it facilitates the Agile workflow in coding environments — and the fact that is has ZERO dependencies means it can run almost anywhere, including Windows systems in future versions, or embedded systems, or inside of a cloud container.

Can your current tool do that with this kind of flexibility or are you muscling Agile into your workflow by team willpower?

I am pleased to announce that Examplar subprocess execution has been reworked and the product now fully supports curses dialogs, ssh clients, and much more — giving you full I/O with any processes that are executed by Examplar.

There is one more feature to implement before the product can hit an alpha release, which is dependency management during the loading/execution of units.  This is mostly just managing the data model which is pretty fleshed out by this point due to the decision to bind design to source.

So excited about this.

Hey all.

Well, it’s certainly been a bit since the last update, but, I have diagrammed the execution flow for the Task class, finally, and fleshed out a (mostly) working implementation of the subprocess execution class consumed during that process.

Note:  Green is cheese for the mice.

Note 2: Not really.  Those are user-configurable items in the datastructure.

Decisions

• At this point implementation is almost complete.  I’ve gone ahead and made an executive decision to include the design documents with the source in the Examplar source repository.
• Any pull requests that do not reflect their changes in the design (where applicable) will be summarily rejected.
• I have removed output checks because it’s unnecessary and complex.  If users desire that they’ll need to implement it in the code that Examplar calls.  It is designed to accommodate that, so there is no loss of functionality there.

TODO

• There is a minor issue with I/O redirection that I’m working on (select/poll madness needed there).
• It doesn’t support curses dialogs very well at all yet, I assume there’s additional communication going on regarding cursor position, refresh, and window size that needs to be accounted for.  I will not consider the product releaseable until curses dialogs are fully supported.  This also applies to interactive utilities like an ssh client.  This was actually partially working until an issue with exit code of child process capture was repaired that ended up breaking it in the process.  Everything comes back to the same class for subprocess execution.
• Dependency check implementation.

Summary of Current State

Examplar is actually a functional product at this point and can be used for simple styles of scripting (i.e. tools that have output but dont require input to STDIN) but not for interactive processes.  The difference between a full, robust software suite and current state is a matter of fixing I/O redirection in a single class, which is in progress.

That should be relatively soon, but in the meantime you should be able to use it to build with your favorite python scripts, bash scripts, ruby scripts, C tools, or whatever and even begin building out things with it as long as those things don’t require keyboard entry from the user.

This is, of course, at the time of writing this, so for posterity’s sake the current source is here.

The repos are kind of the wild west until first release.

Musings

As I’m working through the test scenarios and use cases it’s occured to me how badly I needed this tool in some environments I’ve worked in and some builds I’ve done in the past — yet it’s so simple when you zoom out.  I have a feeling this has the potential for a larger impact than I initially intended, in terms of the quality of the automations that will be produced in this as opposed to just scripting or programming something out without rails in place.

All in all, it’s coming together to meet its originally intended purpose:  A scalable complex (or simple) automation or testing framework.  It could even be used for orchestration or configuration management.

I do have to laugh at the fact that the last 5% of the code took 90% of the project time.

Day 3

My home office has been basically transformed into a war room.

As I was planning it and building it out, it occurred to me that the last two years of my life were hallmarked by perpetually not giving myself enough time to plan for my personal and nonprofit projects.  I’ve been trying to do too many things at a time and not completing any of them.

Enough.

I will complete every project I ever started.  I will complete them well.  I will complete them in a planned fashion.  That is all.

C++ Refresh -> Tenta Rewrite -> Examplar Prep

As I’m refreshing on C++ syntax and design patterns for the Tenta rewrite, one thing is clear:

C++ development is very, very slow.

Examplar, also written in C++, is intended to be done after completion of the tenta client as I found that I had some gaps in my knowledge of the language necessary to complete it, and I did not use a sane design process around either tool due to lack of time during the early stages of both projects.

It’ll be worth going back through the motions though.  I want Examplar and Tenta to both be a single binary, a single config, and in the case of Tenta, a systemd unit target packaged up nicely and only relying on the C++ runtime.

With Tenta complete, I will rewrite Nerve, which will be substantially easier to rewrite, and then I will complete Examplar.

Examplar will be branded independently from SURRO Linux but will be a driving component during the early milestones.

Apologies to the SURRO Linux team for taking so long with this, I’ve been up to my ears in work this year.  Documentation is in draft.

I’ve got so much stuff tah-do..

• New email rules.  I’ve got 36,102 unread emails, 36 unused and outdated email folders whose contents need reorganized.
• Chef setup for all of SURRO and SILO.
• Clean.  Clean.  Clean.  Windex cleans everything.  Except souls.
• Run, Run Run.  Gym every other day.
• Hit the weights.  Lower weight, higher reps.
• Reorganize all my cupboards.  This is serious.  I have vitamins, windex, and power tools in the same storage drawer.  Not kidding.
• Continue studies on a potential rewrite of Tenta in C++.
• Re-read some of the POSIX standard.
• Read ‘Beej’s Network Guide’.
• Re-read ‘Unix Network Programming’
• Re-read some ‘K&R’
• Read ‘Design Patterns: Elements of Reusable Object Oriented Software’.
• Start the formal systems design process for Examplar.
• Build the last POC feature for Examplar.
• Prop up a new Jenkins and mock build server.
• Build a shit-ton of new RPMs.
• Harden tenta SDLC.
• Completely redo the silo landing page.
• Redo the SURRO blog theme.

This content is password protected. To view it please enter your password below:

Password:

Hello, boys and girls!  This is a follow-up to an article I wrote a while back about why sysadmins and developers use packages to distribute software.

On today’s episode of “NO”, we’re going to go over how to create your own RPM repo.  As it turns out, this year I’ve ended up with tons, and tons, and tons of software I want to distribute internally and externally with all these projects and need a clean way of managing it.  I even have a need now for a CICD pipeline, which is a thing all of its own.

This is an often debated topic among some of my circles, where, in many cases, the amount of work in building packages and setting up a repo just frankly outweighs the benefits that packaging software for internal distribution and then distributing from a repo has.

I can certainly sympathize with that, so, I thought I’d list out the steps so that I can keep a record of breadcrumbs along this treacherous minefield of hardships towards:

1. Setting up an RPM repository.
2. Configuring client machines to use that repository.
3. Understanding CI/CD benefits with RPM.

I’ve certainly got my work ahead of me on this one as I’ve heard this is just an enormous and time consuming amount of work.

I’ll have to break it down into smaller pieces I think to get it all done this year.

Key:

Items colored in purple are entry-level system administrator tasks that should be second nature to you by the time you really need to have a custom repo.

Items colored in red are specific to the task of setting up an RPM repository or a client to use an RPM repository.

Items colored in black are mostly sarcasm and general assery.

Items colored in yellow are commands that you type.

Items colored white are the output of those commands you should expect.

*I'm extremely colorblind, so, if I got the shades wrong, let me know so that I can tell you how much I really, really....really care.

1.1 Get a Server

Get a machine to host the repo on.  In my case I used a linode VPS.  This is a straight forward process that I can’t explain for you much more than just to say “Pick a VM size, pick a distro, deploy the image, start the server”.  It’s not very hard.

I picked Fedora 26 for my OS which is a kind of testing distro for next-generation RHEL.  The commands will be identical on any redhat-based distribution, such as CentOS, or Fedora, or Oracle, or whatever you are using.

1.2  Point your DNS at it.

This may be a little more advanced for some but I can’t explain how to set up DNS for your environment.  In my case I used repo.silogroup.org by creating an A record pointing to the server IP, and then setting up reverse DNS on the linode.

1.3  Install Apache.

Now remember on Fedora you’re using DNF now which is the next generation of yum.  DNF has legacy symlinks and CLI options for yum but it’s better to get in the habit of typing it now as opposed to later so that you’re able to transition, and, there’s no guarantee it’ll just be there waiting for you in a future version of your distribution.

 sudo dnf -y install httpd

If you’re on CentOS or similar:

sudo yum -y install httpd

1.4 Create some directories.

You’ll need a place to store your packages.

[phanes@repo ~]$ sudo mkdir -p /repos/{CentOS,Fedora,SURRO}
[phanes@repo ~]$ sudo mkdir -p /repos/CentOS/{5,6,7}/Packages
[phanes@repo ~]$ sudo mkdir -p /repos/Fedora/26/Packages
[phanes@repo ~]$ sudo mkdir -p /repos/SURRO/PRE-ALPHA/Packages
[phanes@repo ~]$ tree /repos
/repos
├── CentOS
│   ├── 5
│   │   └── Packages
│   ├── 6
│   │   └── Packages
│   └── 7
│       └── Packages
├── Fedora
│   └── 26
│       └── Packages
└── SURRO
    └── PRE-ALPHA
        └── Packages

13 directories, 0 files

Ok, so, I lied.  In this guide we’re making five repos instead of one.

This is going to be a crazy amount of work, right?  I always do this, I blow my project scope with additional considerations that’ll save me time later, like I’m planning ahead or something.

1.5  Initialize the repository

Repositories, as should be expected, keep some metadata about the packages they’re hosting so that clients can connect and do the awesome things that package managers do.

The grueling process of initializing the repo database is as follows:

[phanes@repo ~]$ sudo createrepo /repos/CentOS/6
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[phanes@repo ~]$ sudo createrepo /repos/CentOS/5
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[phanes@repo ~]$ sudo createrepo /repos/CentOS/7
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[phanes@repo ~]$ sudo createrepo /repos/Fedora/26
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[phanes@repo ~]$ sudo createrepo /repos/SURRO/PRE-ALPHA
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

Man, this is exhausting.  So at this point all of the databases for all of the repos are initialized.  If you’re tired at this point it’s advisable to take a break, because, like, the rest is really, really hard comparatively.

1.6  Open port 80 and 443 in FirewallD.

This is standard for any installation of apache on a redhat-based distribution so you should be used to it if you’re familiar with these systems already (and here is the code if you’re not):

[phanes@repo ~]$ sudo firewall-cmd --permanent --add-port={80,443}/tcp --permanent
success
[phanes@repo ~]$ sudo firewall-cmd --reload
success

1.7  Symlink the Repos to THE Apache DIR

Before you bring up Apache it’ll need something to actually serve, though.

For reasons I won’t get into here, I’ll just say that this step is needed and do not just plop the rpm’s into Apache’s existing document root.

[phanes@repo ~]$ sudo ln -s /repos/CentOS /var/www/html/CentOS
[phanes@repo ~]$ sudo ln -s /repos/Fedora /var/www/html/Fedora
[phanes@repo ~]$ sudo ln -s /repos/SURRO /var/www/html/SURRO
[phanes@repo ~]$ tree /var/www/html
/var/www/html
├── CentOS -> /repos/CentOS
├── Fedora -> /repos/Fedora
└── SURRO -> /repos/SURRO

….symlinks, yay!

1.8  Enable the Apache Service

Like I covered during my disgustingly over-explained systemd pattern series, we will want to enable the Apache service.

I’m only using one mode and set of configs for Apache in this case because I have a dedicated server for a repo.  If you’re lumping your RPM repos in with an already existing Apache you will probably want to do something different (something else, like, not doing that, amirite?).

Just in case:

[phanes@repo ~]$ sudo systemctl enable httpd

1.9  Start the Apache Service

Riveting!  Just in case:

[phanes@repo ~]$ sudo systemctl start httpd

2.0 Adding Packages

At this point you’re probably wondering “Ok, I’ve got an empty repo.  Now how do I get packages to it?”.

This is up to you and largely depends on how you’re creating your RPMs, to be honest.  In my case, I am using a local VM running a Jenkins instance to build some project-specific RPMs but that’s out of scope for this guide.

I’m also set up to use Jenkins to stage the rpm server which hides all the crazy complex work to do that, but for the purpose of demonstration I won’t be doing that:

2.1 Copy the packages to repo

[jenkins@jenkins rpmbuild]# pwd
/var/lib/jenkins/rpmbuild
[jenkins@jenkins rpmbuild]# scp RPMS/noarch/tenta-0.20171015.91-1.fc26.noarch.rpm repo.silogroup.org:/repos/Fedora/26/Packages/
tenta-0.20171015.91-1.fc26.noarch.rpm 100% 13KB 158.2KB/s 00:00

As you can see, while it may be craaaazy complicated, I used scp to move the package to the repo.

I then issued the command needed to update the repo database to make the package available as seen below.  

2.2 Update the repo Database

[phanes@repo 26]$ sudo createrepo --update /repos/Fedora/26
Spawning worker 0 with 1 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

You really should have your CI server issue this command if you’re using one.  

And there you have it.  You are now a lean**, mean, packaging-ready machine.*

USING THE REPO / Client Machines

I know what you’re thinking: “He said this would be exhausting and time consuming and expensive but that only took like five minutes and only needed one person.  What gives?”.

Here’s where the work is at:

You now need to configure machines to use your repo if you actually want to use it like a repo.  Scary, amirite?

Fear not, bromance***, I’ve got that mapped out for your project too to traverse the angry seas of package management!

3.0 Configuring Repo on Client Machine

In this case I’m deploying super-seekret mind control technology to a server I don’t want people to use so I’m redacting everything you don’t need to know.

3.1 Create a yum repo file

First, create a .repo file in /etc/yum.repos.d:

[ phanes@[seekret-removed ] << ~ >>

[- sudo vim /etc/yum.repos.d/silogroup.repo

 

Second, add these five lines:

[silogroup]
 name=SILO GROUP Fedora RPM Repo
 baseurl=http://repo.silogroup.org/Fedora/26
 gpgcheck=0 
 enabled=1

These are pretty self explanatory.  You’ve got a section header, a name, a baseurl, a gpgcheck, and an enabled.

The section header is what shows up in yum as a handle for the repo.

The name value is the descriptive name of the repo.  This will also show up in yum.

The baseurl value is the actual url for your repo.  This can take http, https, and FTP (and I think some others but don’t do that).

The gpgcheck value deserves being described.  This is a way of ensuring the package integrity by the yum client during download/install.  If you gpg-sign the packages that’s good.  If you host your repo via https that’s also good.  These are both things you should do that I will not tell you how to do in this article (maybe next week?).

The enabled value tells yum on the client machine whether or not to actually use the repo.  Handy, amirite?

So check this out, all ye bromancers of the stones, the demo you have been exhaustingly working your way towards, using a client I wrote in python called tenta that I packaged just for this demo:

4.0 Install the package with yum:

Or do it via a configuration management system which is actually still using yum under the hood.  I won’t be for this demonstration.

[ phanes@super-seekret-removed ] << ~ >>

[- sudo dnf install tenta
Last metadata expiration check: 0:03:50 ago on Sat 21 Oct 2017 09:02:35 PM UTC.
Dependencies resolved.
================================================================================================
 Package Arch Version Repository Size
================================================================================================
Installing:
 tenta noarch 0.20171015.91-1.fc26 silogroup 13 k

Transaction Summary
================================================================================================
Install 1 Package

Total download size: 13 k
Installed size: 28 k
Is this ok [y/N]: y
Downloading Packages:
tenta-0.20171015.91-1.fc26.noarch.rpm 46 kB/s | 13 kB 00:00 
------------------------------------------------------------------------------------------------
Total 46 kB/s | 13 kB 00:00 
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
 Preparing : 1/1 
 Installing : tenta-0.20171015.91-1.fc26.noarch 1/1 
 Running scriptlet: tenta-0.20171015.91-1.fc26.noarch 1/1 
 Verifying : tenta-0.20171015.91-1.fc26.noarch 1/1

Installed:
 tenta.noarch 0.20171015.91-1.fc26

Complete!

Super hard, right?  It took longer to write this guide and the guide only took about 10 minutes.

And here’s where the files from the demo package ended up on the system:

[ phanes@sekret-server ] << ~ >>

[- rpm -ql tenta
/etc/tenta
/etc/tenta/nerve.conf
/etc/tenta/tentacle.conf
/usr/bin/nerve
/usr/bin/tentacle
/usr/lib/systemd/system/nerve.service
/usr/lib/systemd/system/tentacle.service
/usr/share/man/man1/tenta.1.gz
/var/log/tenta
/var/log/tenta/error.log
/var/log/tenta/producer_source.log
/var/log/tenta/tentacle.log

I am also able to determine now if a file has been edited since the package was installed, and much more.

Notes

*In Fedora 26, the default Apache configuration will still show a splash page for Apache at this point.  You may want to add a landing page at /var/www/html/index.html just to polish it out.  In my case I just commented the "Options -Indexes" line in /etc/httpd/conf.d/welcome.conf because I’m using a dedicated server for this and literally only want directory listings.

** Not to be confused with Lean Transformation Model, which destroys organizations, their cultures, costs more money than it saves in most implementations and suppresses talent in workforces amidst a new atmosphere of fear and instability while somehow sounding really cool to stakeholders.  Lean is a westernization of the Toyota Production System used in the 1950s in Japan and doesn’t translate well to western business models (and is extremely harmful to software engineering process), but it did create an entire market of consultancy for how to implement it.

*** This is sarcasm.  This is not hard, not complex at all, and it saves you tons of time.

How I use it

In my case, the servers intended to use the packages are controlled by a configuration management (CM) system. The CM’s manifest has an entry to ensure the latest version of the package from the repo at all times.  This will force the machine to update the package automatically.  This is only for my pre-production environment — I version lock it in my production environment manifest to give me a scheduled release cycle.

For preproduction CI, the Jenkins server is set to listen for changes to my git repository’s master branch to automatically generate an RPM build immediately and then pushes that RPM to the RPM repository, and then updates that repository for consumption by the client.

This way I can commit my code, install my code in a clean way that’s auditable and manageable even if I had tens, or hundreds, or thousands of packages going, I can see the changes automatically in my pre-production environment for rapid testing, and I can accommodate reliable release management with almost no effort at all– well, a little effort — you just read all of it.  Oh– and I’ll never have a single mistake in deployment.

If you’re curious what the jenkins job looks like, too bad, but, I can give you this early version to build on that does almost that.  The filename and version are where some magic happens.  You’ll want to choose between timestamps + build number or git revision numbers.

You’ll have to restructure the order of variable assignment if you want the latter (trust me both you and your devs want the latter but this is a freebie).

#!/usr/bin/bash
set -m
###
# PREWORK
###

# RPMBUILD ROOT CONTEXT
TOPDIR=$(rpm --eval='%{_topdir}')
if [ ! -d "$TOPDIR" ]
then
    mkdir -p $HOME/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS,tmp}
    mkdir -p $HOME/rpmbuild/RPMS/{arm7,arm9,i386,i686,mips,miple,noarch,ppc,ppc64,x86_64}
    echo "%_topdir $HOME/rpmbuild" >> ~/.rpmmacros
    TOPDIR=$(rpm --eval='%{_topdir}')
fi
echo "TOPDIR $TOPDIR"

# TMP PATH
TMPPATH=$(rpm --eval='%{_tmppath}')
if [ ! -d "$TMPPATH" ] || [ "$TMPPATH" = "/var/tmp" ]
then
    mkdir -p "$TOPDIR/tmp"
    echo "%_tmppath $TOPDIR/tmp" >> ~/.rpmmacros
    TMPPATH=$(rpm --eval='%{_tmppath}')
fi
echo "TMPPATH $TMPPATH"

GIT_REPO_BASENAME="tenta"
GIT_REPO="git@source.surroindustries.com:ircthulu/tenta.git"

echo "GIT_REPO_BASENAME $GIT_REPO_BASENAME"
echo "GIT_REPO $GIT_REPO"

# The name of the RPM that is being built
APPNAME="tenta"
echo "APPNAME $APPNAME"

# version number for this build
VERSION=0.$(date +%Y%m%d)
echo "VERSION $VERSION"

# The directory containing .spec files
SPEC_DIR="$TOPDIR/SPECS"
echo "SPEC_DIR $SPEC_DIR"

# the target spec file
SPEC_FILE="$SPEC_DIR/$APPNAME.spec"
echo "SPEC_FILE $SPEC_FILE"

# the workspace dir
prearchive_dir=$APPNAME-$VERSION
echo "prearchive_dir $prearchive_dir"

# the source spec file template
PRE_SPEC_FILE="$prearchive_dir/$APPNAME.spec"
echo "PRE_SPEC_FILE $PRE_SPEC_FILE"




###
# WORK
###

rm -Rf ./*

# create a dest dir (ensuring write access to target fs)
mkdir -p "$prearchive_dir"

printf "\n***\n*** OBTAINING AND HOMOGENIZING SOURCE ***\n***\n"

# Get the source
git clone "$GIT_REPO"

# deglove the unnecessary git repo and child dir
mv $GIT_REPO_BASENAME/* $prearchive_dir &&
rm -Rf $GIT_REPO_BASENAME
echo "GIT REPO $GIT_REPO_BASENAME => $prearchive_dir"

# cat the spec file for the log
printf "\n***\n*** SPEC FILE TEMPLATE ***\n***\n"
cat $PRE_SPEC_FILE

###
## SPEC VARS
## These are variables that are expected to be in the SPEC file.
###
SPEC_VERSION="__VERSION__"

###
# Transform the SPEC file
###
mv $PRE_SPEC_FILE $SPEC_FILE
sed -i "s,$SPEC_VERSION,$VERSION," $SPEC_FILE

printf "\n***\n*** SPEC FILE INTERPOLATED ***\n***\n"
# cat the SPEC file for the log
cat $SPEC_FILE

# bundle up the now distributable source tarball to the rpmbuild SOURCES dir
printf "\n***\n*** SOURCE ARCHIVE CONTENTS ***\n***\n"
tar cvzf $TOPDIR/SOURCES/$prearchive_dir.tar.gz ./$prearchive_dir && 
rm -rf $prearchive_dir

echo "Archive location: $TOPDIR/SOURCES/$prearchive_dir.tar.gz"


printf "\n***\n*** RPM GENERATION ***\n***\n"

rpmbuild -ba $SPEC_FILE

printf "\n***\n*** STAGING RPM to Repository ***\n***\n"

scp $TOPDIR/RPMS/noarch/$APPNAME-$VERSION-*.fc26.noarch.rpm repo.silogroup.org:/repos/Fedora/26/Packages/

printf "\n***\n*** Updating RPM Repository Metadata ***\n***\n"

ssh repo.silogroup.org createrepo --update /repos/Fedora/26

So, I’m working on some new approaches to some projects I’m trying to get off the ground.

One is for a client and I’m ethically obligated not to disclose in this context.  We’ll call that project X.

The other is Examplar, which has attracted some new potential developers that found me through the SURRO Linux project at a Linux convention in Ohio this year.  I’ve got a little time as everyone that showed interest is still getting adjusted to using IRC for reliable communications (it actually takes some time if you’re not used to it).

Since we use IRC exclusively for comms, I’m able to be ready for when they’re able to code in advance.  They’re going to want formal spec and design documents, which I respect and agree with.  Simply having them may even attract yet more devs, so, I need to get off my ass with Examplar.

From there I have got shitloads of additional reading to do.  Namely, a revisit of the puppetlab’s training VM from last year that I abandoned took a hiatus from due to frustration over puppetlab’s tendency to rename already existing concepts.

Meanwhile I’m waiting on news for a C developer role that might actually get me the deep exposure I could only find from having to work an FTE role elbow deep in C and only C and it certainly could be my next gig if they’re into me.  Time will tell.

Oh, found out about the plates thing — its just a fine and a fee for renewal.  They dropped the insurance charge because I found the backup paperwork that didn’t get stolen during the repeated break-ins this year (jesus) since they were kept in the house.

They did get my laptop though since it was in the back of the car this last time.  No comment.

And, of course, being on call is this week.

I’ve got too much shit to do at the same time this year and I’ll probably need to cut a project loose so I can focus on the things I like doing.

This content is password protected. To view it please enter your password below:

Password:

We’re in the process of onboarding a couple of new developers that heard about SURRO at a convention recently.  Stay tuned.

As for current priorities, I’ve been distracted with a non-surro related project and work.  Due to not having much developer numbers to pull from development is slow.

It got brought up that we should have a formal spec and design document.  I can’t disagree with that.  To follow.

Super busy lately.

Balancing work with after-work.

Lots to do at work, lots to do at home.

Need to call the DMV.  They’re up to the same bullshit they tried last year.

I need to renew my plates and pay a fine from not being able to renew my plates on their site (bug?).

As it pertains to ircthulu, I’ve got the listener complete.  It dumps everything to a FIFO that a producer is listening on.  The producer is complete.  The producer pumps to an MQ.  The MQ is set up as well.  From there, a multithreaded consumer pulls from the MQ.

The database is set up and ready to receive data.

The consumer will need to be extended to transform the json payloads from the MQ into a format consumable by the database during pushes to it.  Will be pretty easy with JSON, just deserialize to a json object and populate [an SQL insert statement?] with its values while logging failures.

From there I’ll need to create a restful API that will present data from the database for the UI loader designed in a way to easily extend new types of data to present that is consumable by the UI.

This design will allow the clients, the database, and the UI to all be hosted separately from each other in a massively scalable manner.  Scalability was the pain point for the pilot program.

From there I’ll need to build the UI in parallel to the data tier which is hosted elsewhere.  That’s where most of the work for this project will be, mostly because I’m not a UI person.  Yay angular + bootstrap!

Once the UI is ready for curtains to be lifted, it’s just a matter of getting all the folks who will be running listeners coordinated so that if the client bots are targeted it won’t break the data flow.  Everyone’s on board.

This part is especially important because if, for some reason, someone were to start banning the clients, if I were hosting the clients myself I would potentially face some ramifications with my hosting provider, which I will not risk.

In this model I am simply the provider of a data dumping ground and a publishing point for data that is provided to my system by its users.  I will not be logging which clients are owned by who or keeping a catalog of who the providers are.

Upon my trusty workshop keyboard I code:

The designs my trusty pad hath wrought:

 

In good company my planning was invested:

[never insert this image]

For an untold purpose my world necessitates:

[not my image to insert]

On a single node instance of its back end, with 4 clients pushing data into it, and 8 threads pulling data out of it, a performance test was able to send 500,000 messages in 16 seconds and all were processed on the other end on completion of the sending of the test data and it didn’t seem to be anywhere near maxed out.   It is processing 31,250 messages per second.

That’s roughly 30,000 lines of IRC per second that it can handle without having been limit tested which exceeds the needs for this project.

There is still quite a bit of work to do before the fun can even start with the UI which will be a project all of its own.

Next is the client and the ETL on the other end.

 

…You didn’t think I forgot about this project, did you?

or

How not to sell out by AVOIDING HURTING BOTH your career and YOUR client’s organization

I should be working on other things, I’ve got alot on my plate this week, but I wanted to share this for other contractors who may be following.

I interview regularly.   I get lots of calls from recruiters in my area for IT firms.

This time, it was a company called removed.  There were two recruiters involved.  One was aggressive to get me to give approval for the interview and wanted an updated resume.  The other was aggressive at confirmation.

I send it.  They immediately confirm rate before the interview has even taken place (warning sign one).

Some recruiters are just required to ask so I we move beyond that.  Interview time comes along, and I’m about 5 minutes into the interview for a cloud architect opening before the interviewer at the client says “I see here on your resume you have tech stack im obviously going to learn now experience, extensively, can you tell me a little about the projects you worked on with this?”

I say “I did not put that on my resume because it’s not a stack I’m familiar with”.  He says “yes you did, it’s right here”.

I sense something’s not right because he was hyperfocused on a piece of technology previously that I was not terribly well trained on.

“Hold on.  Something’s not right here”.

So I go to the invite and look at the attachments.  There’s my resume.  I open it….

…It’s been filled full of technology I did not put down, that just so conveniently perfectly all line up with the requisition and have absolutely nothing to do with my background.

Maybe I could have faked my way if I’d looked at the fake resume ahead of time, I’m at home enough with this type of tech that I could probably have performed the role with a little lead time to study up on it.

But here’s the problem:  This industry has a faker problem and it’s an expensive problem.  We’ve all seen them.  They somehow make it through the screening process and make every competent person they work with’s life a living hell for months and sometimes cost millions of dollars to a company before being found out, by which time they’re moving on.  They band together at companies when they identify each other and use politics to suppress talent so that they aren’t exposed.  I’ve seen it.  That, and, the client deserves to know they’ve got a resource provider that is screwing them over if they want to stay in business long.

I say “Yep.  I knew it.  I’m really sorry sir, this is not the resume I sent over.”

“It’s not your resume?”

“Well, it is, except, the resume I sent over had different specializations on it.  I am so sorry about this, I am going to have to end the interview and then fire the firm I was working through for this interview.  I will never work with them again and you should be made aware that this has happened.”.

“Ok.  Thank you for telling me.  I’ll go ahead and alert my superiors to this.”

“This is unacceptable.  You’re looking for a removed architect.  I have strong specializations but would not be a fit for this role as it’s not what I specialization in”.

“Yes, I agree.  Well Chris, I hope to see your name on future openings here.”

“You too as well sir.  I’ll be connecting through a different company if I get a chance”.

And we ended it on a good note.

It was hard to do.  That job could have doubled my income.  But it’s not just the right thing for you that needs considered during your career– as an engineer, or an architect, or a developer, you, as a consultant, have a set of obligations to your clients and to your career field.

The core foundation of those obligations revolves around integrity and in this case — cleaning up the industry in some small way.  A company that was being exploited by one of their vendors now knows to look out for dishonesty from that vendor.

Just like you’d pick up an empty soda can and throw it away at the bus station.  I’d expect the same from a candidate that I’m interviewing.

This content is password protected. To view it please enter your password below:

Password:

A more readable journal can be found here.

[ phanes@prime.surroindustries.lan ] << ~ >>

[- cat 2017-09-20_rpm-build.txt 
@@@ rpm build instructions
@@ build an SRPM

@ 2017-09-21

/*  Note:

    An SRPM is a .src.rpm file that contains the SOURCE for an RPM
    in most situations this will also include source code for the 
    application and this is compiled during RPM generation. 

    Pipeline for an SRPM:
	$PROGRAM SOURCE, SPEC FILE -(rpmbuild)> $PROGRAM.src.rpm -(mock)> $PROGRAM.rpm

*/

$ sudo dnf install fedora-packager rpmdevtools
$ rpmdev-setuptree
$ cd ~/rpmbuild/SOURCES
$ git clone git@github.com:cmpunches/Marco.git
$ cd .. && mv Marco/ Marco-1.0 && tar  cvzf Marco-1.0.tgz Marco-1.0/
$ cd ~/rpmbuild/SPECS
$ rpmdev-newspec --macros Marco.spec
/* output
Marco.spec created; type minimal, rpm version >= 4.13.
*/

/*  Note:
    Creating the spec file is actually careful work that is error-prone and requires training and testing.
    You are advised to read the rpm documentation, particularly pertaining to SPEC file SECTIONS and MACROS.

    Note:
    This is much easier with a build system, either as a result of a cmake build or in a larger system like Koji.
*/

$ vim Marco.spec
/*
Name:           Marco
Version:        1.0
Release:        1%{?dist}
Summary:        An HTTP echo service.  Returns the request it received to an HTTP client in a consumable format.
License:        GPLv3+
URL:            https://github.com/cmpunches/Marco
Source0:        %{name}-%{version}.tgz

BuildRequires:  cmake boost-devel
Requires:       boost-devel

%description
SURRO Industries' HTTP echo service.  Listens on port 1999, and replays requests to the requesting browser.

%prep
%autosetup

%build
%cmake .
make

%install
%make_install

%files
%{_bindir}/marco

%changelog
* Thu Sep 21 2017 Phanes <punches.chris@gmail.com>
- Initial version of the package

*/

$ cd ~/rpmbuild/SPECS/
$ rpmbuild -ba Marco.spec

/* compiles and bundles into an SRPM */


@@ Build the SRPM into an RPM

/* initial mock setup */

$ sudo dnf install mock mock-scm
$ sudo usermod -a -G mock Phanes
$ newgrp mock

/* end initial mock setup */

$ rpmbuild -bs Marco.spec

/* Note:
   At this point you've got an RPM and an SRPM for Fedora since I'm on a Fedora system. 
   Probably not desired.
   Mock allows you to compile for other systems.
*/

$ mock -r  <Tab complete for system profiles>
$ mock -r epel-6-x86_64 ~/rpmbuild/SRPMS/Marco-1.0-1.fc25.src.rpm

/* Note:
   Tons and tons and tons of loading.  That's expected.  It's creating a mock build environment for the
   compile so that you're getting a clean package that's aware of the profile of the system it's intended for.
   These packages are not installed on your driver.

   Note:
   Due to a transition from yum to DNF for RH upstream, which has not yet hit EL6 and EL7, you will need
   a newer version of mock that fixes a bug caused by that change.  It is in the updates-testing repo.

	sudo vim /etc/yum.repos.d/fedora-testing-updates.repo

   Enable it, then `sudo dnf -y update && sudo dnf update mock` 
   Note:  I also had to install the undocumented package yum-utils to bring the /usr/bin/yum-builddep tool back.

   At this point I had to reduce the boost requirement to 1.4 to compile on EL6 which won't compile without a
   full rewrite of my solution since it heavily uses the boost libraries.  EL6 is old.  Trying EL7.
*/

$ mock -r epel-7-x86_64 ~/rpmbuild/SRPMS/Marco-1.0-1.fc25.src.rpm

/* EL7 is also too old for the version of boost used in the application code.  Will have to revisit this for EL distros. */

@ 2017-09-21_2

/* Let's go ahead and bundle the boost dependency too. */

$ cd ~/rpmbuild/SRPMS
$ wget http://download.opensuse.org/repositories/home:/aevseev/CentOS6/src/boost-1.63.0-1.9.el6.src.rpm

/* generates a spec file */
$ rpm -i boost*rpm

/* compile to FC package */
$ rpmbuild -ba  boost.spec

/* yay, more dependencies: 
[ phanes@prime.surroindustries.lan ] << ~/rpmbuild/SPECS >>

[- rpmbuild -ba boost.spec 
error: Failed build dependencies:
	libproxy-devel is needed by boost-1.63.0-1.9.fc25.x86_64
	openmpi-devel is needed by boost-1.63.0-1.9.fc25.x86_64

$ cd ~/rpmbuild/SRPMS && wget http://fedora.inode.at/updates/26/SRPMS/l/libproxy-0.4.15-2.fc26.src.rpm

$ wget http://mirror.math.princeton.edu/pub/fedora-archive/fedora/linux/updates/testing/24/SRPMS/o/openmpi-1.10.7-1.fc24.src.rpm

$ rpm -i libproxy-0.4.15-2.fc26.src.rpm 
$ rpm -i openmpi-1.10.7-1.fc24.src.rpm 

[ phanes@prime.surroindustries.lan ] << ~/rpmbuild/SPECS >>[ phanes@prime.surroindustries.lan ] << ~/rpmbuild/SPECS >>
[- rpmbuild -ba libproxy.spec error: Failed build dependencies: libmodman-devel >= 2.0.1 is needed by libproxy-0.4.15-2.fc25.x86_64 pkgconfig(javascriptcoregtk-4.0) is needed by libproxy-0.4.15-2.fc25.x86_64 pkgconfig(libnm) is needed by libproxy-0.4.15-2.fc25.x86_64 pkgconfig(mozjs-38) is needed by libproxy-0.4.15-2.fc25.x86_64 python3-devel is needed by libproxy-0.4.15-2.fc25.x86_64

 

There’s a better way to do this I was just hoping to have a clean build tree for boost.  I’m just too fucking tired, I’ll have to carry this over to a third day until I can get some time to myself.

Moar diagrams

This time covering service state control patterns:

So from here I’ll need to create an outline, and then condense everything into brief, concise writeups that fit into their appropriate sections.

As for service controller antipatterns, I just don’t have alot of those besides what was already covered.

It’s going to be difficult to justify the reasons why in these, so, I’ve got a good couple of days more work ahead of me.

Update:  These need a change — variant A and B for established control patterns needs updated to reflect that it requires a root setup for enabling user services but does not require root or sudoer to actually consume.

Speaking of Work

In other news, I’ve hurt myself at the gym.  I have some DOMS after hitting curls too hard too early and can’t fully extend my arms.  I’m basically a whiny T-Rex until it’s healed up.  I had to actually call in and sleep for a good day and a half.

Welcome back.

I said I’d post this “tomorrow” in my last post, and it definitely has been several “tomorrows” since then (mostly because I’m lazy and busy, which is a terrible combination), so let’s recap:

This series of articles revolves around:

• The reasons why packaging any software used in your project’s environment is important.
• Service management considerations for your software on the Linux operating system.

The approach we’re taking emphasizes where this would take place in the SDLC, specifically the ALM (Application Lifecycle Management) which I encourage you to read up on if you’re not already familiar with these entrant software engineering concepts.

To do this, we’re removing all the coding, all the solutions architecture and design out of the software engineering process so that we can look at just these two aspects — and the example that we’re using is an HTTP echo service that I’ve hacked out of a very simple HTTP server in C++ with the boost library to create something called a microservice if you’re familiar with SOA (Service Oriented Architecture).

These are all suggested google search terms.

But first, what genre is this even?

Anyway, to bidness, some patterns and antipatterns as it pertains to package management (I’ll be breaking this into cleanly organized content when it’s all laid out btw), I’ve started some crude graphs that demonstrate what I’m actually saying to be used in a more meaty article later.

Package Management Is a Fundamental Best PRactice

First, we’ve concluded that packaging your software is an intrinsic result of abstraction of software to units on your system using a simplified representation of parts of the OSI model:

Service State Management Patterns and AntiPatterns

Patterns

Variant A and B

Antipatterns

Examples A, B and C

Given the amount of research it takes to justify the reasons why sysadmins are trained to do this in this way, I’ll need to spend more time on the meat of the article and will end up condensing this whole series into a whitepaper.  But for now, I need the rest of my Saturday.

Howdy.  My name is Chris Punches (vanity link), and I’m here to talk to you a little about the planning around the Deployment Phase and Maintenance Phase in the Software Development Lifecycle.

# BEst Practices – I want to Believe

To preface, over the years I have seen good habits that save lots of work for everybody, and I’ve also been brought in a time or two after damage has already been done by bad habits — often either for the sole purpose of cleaning it up or making recommendations on what to do. I’ve also dispelled some bad habits that “everyone was doing” and those are plentiful in the industry as well.  Basically, I’m a software, systems and infrastructure janitor, and I try to share what I learn where I can or document my own learning for others.

Let’s talk about the SDLC, but let’s remove the SD from it

So, for this example I’m going to demonstrate by removing the parts we already know about from the SDLC: The actual design and development of the application. While it would seem at first glance to be important to talk about software development in a talk about the Software Development Lifecycle, that part of the lifecycle actually so dominates most environments that conversation about the rest of the lifecycle can get drowned out:

In this example, I want an HTTP ECHO SERVICE that will simply return 200 to a browser along with the headers supplied in the original request. We’re going to skip design and coding, and omit getting it in version control but assume it will be and that all best practices in software engineering were followed up to this point.

Note: This will nicely compliment my proxy discovery tool that I’ve recently created. It allows you to verify that a proxy on the list is not tagging users with an injected header (something that will certainly compromise the identity of users of that proxy if not checked for before using it).

So in this case I got kind of lucky right after I started and found that the boost library documentation in C++ has an example of a very small HTTP server, that I just kind of hacked up to regurgitate whatever request was supplied to it instead of sending an HTDOC root path and doing all that fun stuff.

Annnnnd done:

Then an hour of goofing around with gitignore, building the readme, creating the public repo, and we’re at about the point that too many folks stop developing because they think software development only involves the development of software. I actually enjoyed building this so much that I’m going to host it as a permanent service at, say, proxycheck.silogroup.org and I’ll put the source here.

# How to get from “it compiles” to “I GET PAID TO DO THIS”

So here’s the meat of what I’m writing about. The software itself is only one piece of a much larger set of interacting components in your SDLC, and some of those components are the people involved, as well as the time they spend on it, and all the things people have to do to coordinate with that in an operations environment. If I were, say, hosting it in an organization (which I am, SILO and SURRO), then I would have considerably more things to consider than if I were just testing it on a VM or showing a friend.

That’s probably all some folks are used to besides a “sudo make install”, which you can get by with on a disposable machine, but not for actually hosting a service reliably, and will get you shot if you do in enterprise.  That’s almost as bad as compiling on one machine and just copying the assembly over (READ:  PREPROCESSOR, COMPILER, ASSEMBLER, LINKER, LOADER).

If you are out there reading this and writing software, then definitely don’t stop here because this is where it gets cool.

# So how do we get there knowing at least *someone* will want to:

• make updates to this service’s source code that I will want reflected in production
• update the installed version to newly compiled versions after that
• troubleshoot it in familiar, reliable ways when it breaks (assume bugs exist)
• report on the assembly metadata as supplemental process to that
• control the virginity of files associated with that piece of software (hehe i said ‘virginity’)
• control the service’s state, particularly during installation, upgrades, and removal of the software (too many people forget that last one and end up breaking their systems when they’re first learning stuff — think:  library conflicts).

These are great initial considerations if you actually care about what you’re doing or if what you’re doing involves other people.

# Enter the OS (+Userland)

First you have to understand that you’re controlling a piece of software’s lifecycle, and that software requires a platform: The Operating System.  It is the platform upon which software runs.

This is where alot of people get tripped up because it’s easy to forget what the purpose of things are in complex systems after we’ve seen them for a while and don’t use them.

The operating system is the interface between the user and the hardware. Everybody knows that.

Here’s what alot of people don’t know though: It consists of a kernel or userland components.

Userland is everything on the OS besides the kernel.

Userland is divided up into three categories: (on Linux Systems)

• The C standard library,
• Low-level system components,
• and user applications.

It is similar on Microsoft and Apple systems.  System daemons running, say, systemd, logind, PulseAudio, and other low level stuff fall into low-level system components. User applications will be things like your browser. Apache is a system service. Tmux is a user application. We’ve built a system service because:

1. It’s always running.
2. It’s meant to be a background process.
3. It’s unix-like.

Learn more about daemons, or services here.  The criteria for a system daemon are well defined and understood and I have no need to justify them here.

You don’t handle these processes that meet the needs discussed in the previous discussion in the same layer as your application if your operating system will be running other software that other people write and support.  But why?  For the same reasons you don’t leave a mess in the kitchen when you have roommates first of all, and in the case of organizations you’re saving time and money due to uniformity of process (oh shit now I have to justify why uniformity of process saves money — ya know what, fuck it — it does, just believe it for now) and reduced human error (it is a form of automation to behave in this manner but it is so common now that people don’t even realize it). But, mostly, it’s because of the importance of abstraction:

# Wtf is abstraction?

If you’re a developer, it probably means close to what you think it does, but is not an inherently development-centric term — it’s also an architectural term (and has a similar meaning in several unrelated contexts):

“In software engineering and computer science, abstraction is a technique for arranging complexity of computer systems. It works by establishing a level of complexity on which a person interacts with the system, suppressing the more complex details below the current level.”

So, in this case, there are some revised points of consideration to point out that are solved by abstraction of the software artifacts:

• We want this service to be portable to a wide number of machines ( this allows a rational distribution model for this piece of software and also prevents conflicts with other software on the providing platform )
• We want to be able to deploy, update, and remove cleanly ( this reduces outages of my new service before, during, and after updates and installation )
• We want the above to be done in a uniform manner to other software operations on those same machines ( this reduces human process surrounding unit management on the target platform, allowing one process to exist for all software in a way that’s able to be controlled and improved; this also reduces human error by natural consequence of that, reduces the amount of support training needed on this service, etc.  )
• We want to minimize the amount a human hand has to touch the unit or the platform serving the unit.  Be easily updateable without potentially breaking things or creating inconsistency in the environment we’re installing to –not touching too many “things” during operations involving it ( this should include maintenance of whatever mechanism or automation is being used).
• We want to be able to audit both the assembly and metadata of all associated files during troubleshooting and analysis, even planning, and this metadata should include at minimum version number, dependency trees, install date, and more if we can get it ( while more complex than this, it basically allows us to know

So we need to Abstract it into a unit.  How does this work?

Artifact abstraction in computer science is widely regarded as packaging the software, and sometimes in several layers of packaging.  The definition of packaging is literally abstraction of a software unit.

If you’re in Javaland you’ve probably used Artifactory for this.  If you’re not, you’ve probably used MSI if you’re on windows, or RPM if you’re on CentOS or another redhat derivative, or DEB/APT if you’re in Debian or Ubuntu.  If you’re on Arch Linux you’re probably using pacman.  Gentoo uses emerge (it technically is package management even if it doesn’t feel like it in gentoo).  There are distro-independent package managers of varying qualities.  I’ll not mention them because they generally tend to bring instability to systems due to bugs.  If you’re on OSX apple has their own package management scheme, it’s actually so seamless that it is barely noticeable — however they also have alternative package managers like brew.  If you’re on a freeBSD system, you’ll probably be using pkg_add which is built-in and portsnap.  If you’re on SUSE it’s zypper unless that’s changed (been a few years).

In my case I want to target for Fedora since that’s the distro I use on all my servers currently.

In the case of a robust package manager like rpm or dnf, the process of installation generates associative metadata in a repo for every file in the package meeting every need presented but one.

I’ve actually rarely had to justify package management though, it’s really such an introductory concept and is so fundamental to software management that almost invariably doesn’t need justified.

This is not an operating system concept though — it’s actually a software engineering concept, a fundamental one, that the OS makes heavy use of.

Reiteration:  The Linux Operating System does not provide a package manager.  The project that develops the package manager provides it to the distribution maintainers who decide to include it or change to another package manager.  This is easily confused because often the team for a package manager that is widely used by an OS works in close collaboration with that distribution provider.

On to Daemons and Services

So, I didn’t solve every need that I have for the actually very important “people” part of my SDLC:  I need a uniform way to control service state for all system services, not just for this one piece of software, but for all software I put on that system.  I don’t care if it’s in the base packages the distro provides, or if it’s something I put on after the fact — I’m building a complex set of IT Services out that I want to have clean management of and this daemon is no exception.  Since in the case of SILO and SURRO the same people who manage the state of this service will also be managing any network services or anything else on the hosts, it’d be much better to have it all using one tool.  If I don’t have a uniform state management facet of this service’s lifecycle, the silo staff and surro staff will be running around pulling their hair out, being well trained on linux systems, trying to figure out how to start which service with which methods.  After about 10 or 20 disparate services like that, it can become almost unmanageable even if you take copious notes unless I allocate staff to only watch those services and nothing else (and I can’t allocate people to just OS maintenance because my application layer is deeply dependent on it and I need them to support that user application layer as my core organizational goal).

But first a word on service state.

# Service State is a Lifecycle Too

If your daemon is running, that’s a state.  If it’s not, that’s a state.  Here are all the states I care about:

1. A Running Process,
2. No Running Process,
3. A boot-enabled process,
4. A boot-disabled process.

Services are crazy simple when you look at service states.  Just like processes.

Now sure, the application itself may have many states within it, an uncountable number of states — but those are up a layer in this model and I don’t care about them yet.  At this point I care about the state of the service itself.

Oh, I should also point out– a consistent method for reporting on the state of the service is necessary as well, as during troubleshooting I don’t want to wonder at all which of these four states my process is in (note: not all of these states are mutually exclusive) if other services will depend on it (and they will in my case).

So how do you bring continuity to service state management?

I’m glad you asked, very large header text.

It starts by understanding how your software is launched.

In all cases, including Windows, and Linux, and OSX, and BSD, your server starts with a single process bootstrapped from BIOS/UEFI.  That process is called init.  The kernel itself executes init as the bootstrap mechanism with a hardcoded path to the init program.  It gets complicated and divergent from here but basically init calls other processes.  In the case of Fedora, it launches SystemD, which then starts the system’s services that allow it to become multiuser, and sets up various features that most people don’t dive very far into.

From there you log in, and do things.  Kinda.  I skipped over alot of stuff, but that’s the gist you need for this.

Once it’s able to handle users, I can log in and use it’s reporting and control tool, systemctl to:

1. Start a service that is not running or stop a service that is running.
2. Report on the service state of any service.

And there you have it.

So, if I want a uniform way to manage system services I would plug right into what’s bringing up the rest of the daemons running on that server.  Even if I don’t — by proxy of another process I still am anyway — it is not something you can opt out of.

• We want to be able to control the service’s state cleanly.

And for the final point:

Oops, changes of service state with systemctl require root and I don’t want to give root to anyone.  SO DOES INSTALLING PACKAGES.

This is a perfectly valid concern.  As it should.

So, we’ve made it this far only for SILO security staff to blow the whistle on my design because SILO’s servers require more than one person to have access.  What will I do?

 

Well, here’s where it can get a little complicated but it’s definitely a solved problem.  Since I’m not incompetent I set up all SILO staff to have users in a shared group called “staff”.  Granular access control is the point of usergroup administration.

Then, I divided the staff into users who can install software and other users who can manage service state.  One subgroup is “installers” and one subgroup is “sysadmins”.

Then, I added two lines to a frequently misunderstood file called the sudoers file.

%installers ALL = NOPASSWD : /usr/bin/rpm
 %sysadmins ALL = NOPASSWD : /usr/bin/systemctl

And from that point forward, my staff who installs things has the access they need without root.  And my staff who controls service state have the access they need, also without root.

Or,

I could have them use rpm to install to a shared system path that they do have access to, like /opt/SILO/RPM_ROOT and force approvals for service scripts, though, that can get very complicated quickly.

In my case, I opted for the sudoers entry because it’s very easy to manage and sudoers leaves an audit trail with logs that I have pumping into my monitoring system and SURRO pager.

Even better news:

Packages are as easy to build as whatever software I am packaging is.  This is also usually automated, and usually in a CI-like system.  Check out koji or git-build-rpm or Jenkins if you like getting your hands dirty with the build process.  Having a package regenerated when I do a git commit is excellent as it can immediately reject if it doesn’t compile.

Systemctl is stupidly simple to use.

I hope that this report saves readers a great deal of money and time — from reallocated time spent automating things, to outage prevention, to general system stability and reduced error rates.

Implementation

In my case I’m doing a one-off so I won’t be using a CI model although I will be eventually for SURRO.  With this package I’m going to just create the SystemD script, which is all of about 4 lines, build an RPM, install that, and then open the firewall for my new webservice.  I unfortunately won’t have time tonight to do that though, so it’ll have to be tomorrow.  This should cover all the important parts of the lifecycle you need to know — I don’t need to document very much about my service now since it’s all using tools that I’d not let someone be staff at surro without knowing how to use what I mean by that is that while the explanations and reasons behind these choices may have been rather complex, the practices that were mentioned as a result of that thinking are entrant level exposure.

So, I’ve been lazy.

Apparently during the move to HTTPS, I failed to set up some things.  The personal blog is still loading http for media assets and RSS, but http is no longer available, and one of our certs expired because i didnt properly configure the alias for the automation behind cert regeneration and then also didn’t set up a monitor for that.

I’ve work to do.

NEWS Outlet is now Mobile Friendly

Did give the news outlet a facelift to make it mobile-friendly, though, so, there’s that.  It’s still http though.  Gotta get on that.

TODO

• move news outlet to phanes.silo server
• set up automated cert renewal for news.silo
• regen certs for phanes.silo and news.silo
• fix https in application layer for phanes.silo
• update news.silo to point to phanes.silo
• set up LDAP auth and UID/GID mapping for SILO

LDAP for SURRO and SILO

I’m also in the process still of finishing up configuration of a new LDAP service for everything.

So, it’s SSSd, NSSwitch, and PAM, yay.  3 huge parts of the linux system I never really made myself learn.

I think starting out with Kerberos 5, SLAPd, LDAP working my way down the docs is the smart way to go there.

The fun part is Ipsilon after that but I want to get system auth up and functional first.

Reading for Network Login SErvices

So, when it comes to KRB5, SSSD, NSS, and PAM, this is an area of linux systems that everyone skips over (myself included) because there are just so many moving parts and it’s a pain in the ass to troubleshoot.  But it needs learned if we’re going to implement it in a new distro.  We’re looking more and more like an enterprise infrastructure every day and I need to get us to “that magic quiet zone” after all the shifting around.  I find this part of the system to be far more complicated than SystemD and even compiling a custom kernel.

• The TLDP Guide for Kerberos.
• RFC 1510 for LDAP definition.
• An Ubuntu Guide (I’m using Fedora but it’s helpful) for Kerberos + LDAP.
• The official MIT-origin Kerberos Documentation
• SSSD and NSS docs from Red Hat
• SUSE’s expectedly sexy and clear presentation on SSSD.
• An introductory text on how PAM works from Tux Radar.
• The official docs on PAM.

Examplar

Still plugging away.  Pipes, pipes, dupes, forks and more pipes, oh my!

New Rack SErver and 32bit compat in python

Someone donated a new rack server for lair.silo.  I’m not sure what I’ll be doing with it besides hosting auth services since it is 32bit and can’t be the archipel demo (and can’t really be preprod for any of the python services since 32bit py environments don’t seem to have library continuity with 64 bit in just the right cases).

Oh well.  Tons of shit to do.  Time to go work out first and then get to it.  With it being labor day weekend it’s the perfect chance to get caught up.

Wow.  What a great vacation.  It’s been, like, forever.

In terms of stuff that would be relevant for this blog, not much.

Well, I did build an interesting little tool for a browser anonymization experiment:

HOWDI – Http Operations With Dynamic Input

https://github.com/cmpunches/HOWDI

Basically is a stripped down rewrite of SlitScan only a little more usable (and maintainable — there were serious design quality issues with SlitScan that I found and wanted to just reimplement almost all of it).  Needs a posix interface though.

How HOWDI works

It fetches proxies from open proxy lists.  It then checks from a random selection of IP address echo services using each proxy in the list with managed concurrency.  It then creates a listener on localhost that routes to each proxy until a side disconnects.  It is tested and working but there’s a pretty gnarly file descriptor leak somewhere in the socket code for the L4 proxy portion.  I’m not entirely sure where yet.  Socket programming is a big gap for me, so I look forward to researching it.

Joined a Gym

I decided on my awesome vacation that I’m spending too much time in front of a screen.  So, I joined one of those super fancy ‘everything in one’ gyms.  Feels like a cross between a Y and a country club.

That first workout was rough.  It’s always the cardio that gets me.  The pool and steam room were a nice change and the building is absolutely gorgeous inside, where the staff are fantastic.  It’s like a hotel.

Migrated this server

I migrated the blog and some various IRC projects over to a new server.  You’ll notice in the URL when you go to blog.surroindustries.com it now is a CNAME entry for phanes.silogroup.org.

Examplar

Still pecking away on how I want part of process execution to work.  It’s weird with this project so much more time is spent on design than coding.

Ivory Tower

I’ve gone ahead and released Ivory Tower into alpha.  I don’t think I ever announced it.  Ivory Tower is a GTD system for Linux desktop written in GTK and Python.  Feel free to leave pull requests or requests for changes, or file bugs.  A bit rushed but everything is.

IRCThulu

Designs still in draft.  I’m not sure how I want deduplication to work yet.

And a bunch of other stuff I probably forgot about.

 

Not a whole lot of updates, I guess.  I’ve just been working alot.

The last week or so I’ve been researching cryptocurrencies since it’s been a bit of a gap for me.  I have an idea I’d like to try out using microtransaction calculations on cryptocurrency exchanges based on predictive trend convergence when I can decipher GDAX’s or some similar Forex-type hub’s API well enough to create trade logistics algorithms.

Then there’s message queues.  I’ve long talked about implementing my own very simple message queue.  But, I think it’s time to get a little more experience with existing MQ models so I can see if there’s any feature bloat I may actually care about whenever I do find time to do that.  To be honest I can’t take on any more pet projects or I won’t get anything done.

Examplar subprocess execution is fleshed out but not yet implemented.

There was a suggestion to use Yocto for Foster but I’ve decided not to go with it as it would push all the real work from M2 to M3 and the type of work it is needs to be burned in by M3.  It certainly is on the radar for projects to learn from though.

And then there’s also crushing my enemies, seeing them driven before me, and hearing the lamentation of their women.  Work in progress.

 

 

I am pleased to announce the beginning of a long-mentioned, but previously shelved project: IRCThulu. This is a product that was demonstrated during its proof of concept phase in August of 2016.

This project is able to serve as a means to providing a very informative resource to the public as a source of unending Q&A in discussion of software development, linux systems administration, and much more — between developers and engineers— as they develop and do complex tasks that your team may be facing.

IRCthulu is generally oriented towards IRC operators due to its unique identity matching features that will allow you to track users in ways not generally available today, and is not intended to be a public site hosting tool, though it certainly could be if you’re wanting to make your IRC channel public.

I am now allocating all spare funding, time and resources to the development of IRCThulu from here until the foreseeable future. While SURRO Linux will be getting top priority still, I’ll need stuff to pick away on as a distraction.

Bear in mind that IRCThulu is only developing the components necessary to accomplish the capabilities it provides and that other parties have committed to hosting it when it becomes functional.

That in mind, as part of phase 2, as of today, July 31st, 2017, all content on all public python IRC channels sponsored by the PSF are now to be considered in the public domain. It is advised of any channels with any privacy concerns caused by this to alert their users regularly and immediately that their IRC channel, including statements made by users, and their connection metadata, are now published data in the public domain and are google indexed. The reason for the importance of this is that with the introduction of this project they are moving from a closed, private mode to a public one.

Any member of the PSF is welcome to reach out should they have any questions, concerns or comments.

Development begins immediately. This will be a closed source project until the goals are met, at which time it will be released with a GPL license along with the source.

365 Days of Total Transformation

Man, what a crazy and productive year this last year’s been.

The career changes I made a year ago and the extracurricular strategies I formed have started to burn in enough that it’s completely transformed my home life, my work life, my general outlook, and desire to be surrounded by puppies, for the better in a much needed way.

I’ve broken a deep, decade-long depression in the process and I can not be thankful enough.  Groceries from Whole Foods.  Vitamins, exercise, sunlight, diversity in activity, respect among my peers, and consistent growth.  It was that simple.  Oh, and new furniture.

New furniture helped.  It was weird how big of a difference that made.  There is something about that I don’t fully understand.

I’m ALSO Taking Downtime as an R&R supplement

I have to confess, I’m in a “downtime” mode right now and I’m so not sorry.

I did manage to get to an alpha state for Ivory Tower.  It is not quite ready for release but serves as reasonably accurate prototype for a C or C++ rewrite where the library bindings are little more sane.

I just got back from a python convention, which I missed most of unfortunately, this year, due to being on an outage call for my primary client.

Speaking of Python and Bullshit

And now I will indiscriminately throw lots and lots of people under the bus.

Speaking of Python, being reasonably accurate, being sane, etc, etc, etc, my attempts at negotiating for a reasonable compromise with the PSF around IRCthulu seem to have failed after a year of extending innumerable courtesies to reach a sustainable agreement.

I don’t know what’s going on with the PSF these days, but I’ve never seen the kind of behaviour from a respectable organization that I’ve seen surrounding the currently shelved IRCthulu project as I try to establish a reasonable relationship with that team.  I am not impressed with their ethical perspectives and I am easily impressed in that area.

It seems I am not alone in this regard.  It is concerning.  I think that I’ve reached a point where we may be at an impasse in negotiations unless they can prove to me that they are sane fucking people able to have sane fucking thought processes.

I’m at a point in my life where I’ve got just zero patience for manipulative assholes in my mindspace anymore. Between Red Hat’s oppressively idiotic influence in the open standards area and PSF’s absolutely pretentious views from their IRC teams, gnome’s arrogance in the Ux and DE crossover compatibility areas, AMD’s archaic and mostly retarded stance on chipset instruction disclosure in combination with their inability to package their own fucking software for their target platforms,  Apple’s poisonously destructive propaganda surrounding their efficacy as a provider of sane development environments, Oracle’s refusal to die or shelf Java, IBM’s and HP’s incredible ability to repackage open source systems in an unmaintainable and unnavigable way to secure support contracts behind the veil of lack of client training, and the continued fucking existence at all of lotus notes I’m just remarkably disappointed in the times I live in sometimes.

Then I see articles like this.

About the machine learning algorithm that used improper English to shorten sentences due to a lack of proper constraints being set, where it is labelled “AI”, given spooky undertones to misinform viewers about the nature of what was learned (which wasn’t really learned as they already know the language constraints need calibrated to avoid this).  Some of the downstream publications about that project allude to it being “shut down in fear of its capability”.

And it’s utter horseshit.  Nobody who understands how those tools are built thought that.  Nobody involved in the project thought that.  It’s just a lie.  It’s not really AI.  It’s a borked function call to a neural net for predictive decisions.

This concludes my rant about the moral and cognitive erosion in my field of passion.

End of Bus's Danger Zone.

But most of All, I hate Arrogance, especially Mine

And now I will throw myself under the bus.

Anyway, out of all the stupid things I see, arrogance is the worst one, and probably the one I dislike to see the most because it is so easy to fall into.  Other peoples’ arrogance rubs off on the people around them and it can be so infectious.  I’m guilty of this.  I think this is something that anyone working in the software engineering or IT field can fall into because it is just so prominent.  I actively try to catch myself before I let it out, but sometimes it gets out and then I feel like an ass for a while when I realize I’ve been arrogant.  I did that last night I think after 2 too many beers knowing I can’t drink anymore for health reasons.

I think the reason this field is so prone to that is based largely on the fact that it is such a technical field.  The levels of complexity that developers and system administrators deal with on a regular basis gets normalized in their worldview, and there is a wide range of skillset and capability among performers.

There is also a shadow of a problem of pretenders or fakers that shows up occasionally and I think this is the complicating factor; I myself often during screening spot fake resumes or fabricated knowledge during interviews, and have even found some in the past after they were brought on, frequently enough to be a little on the paranoid side, which is both a positive thing and a negative thing– I also spend alot of time on IRC which is an absolute cesspool of human hubris.  There are two points of consideration over why this is a negative thing to me currently:

1. Everybody’s got gaps they’re actively trying to fill.
2. Gaps don’t mean someone isn’t competent in their field

You literally can not learn it all in this space if you spend two lifetimes studying it.  Where one person’s strengths overlap another person’s gaps I think is where the most likely place to see arrogance surface is.

Dev vs. Sys is my favorite example of this because it’s the easiest to describe for me.

Developers can be amazing developers but maybe have some gaps about operating system or deployment architecture depending on what type of developer they are, and System Administrators may be incredible sysadmins who run a clean shop but maybe aren’t so great at development.  DevOps is kind of in the middle by trend, a little of both but not quite as deep in one or the other.  Developers and Sysadmins often are going to have entirely different approaches to solutions engineering or architectural approach to solving a problem, and that should be expected.  This was really highlighted when the devops concept started being ingested by large corporations.  I think often that some devs will not see that or some sysadmins will not see that and it can create misperceptions that create assumptions and ultimately an air of arrogance towards the other.  I got lucky and ended up at mostly great organizations with cultures that none of this applies to, but I’ve seen it before.  When people disagree, everybody needs to be able to be wrong and that’s hard for very smart, technical people to be when their livelihood depends on their expertise, credibility, and ultimately, their delivery of product.  A personal brand.

That, and early IT culture was a little rough around the edges.

Most large orgs in general are aware of this to some degree and have done well in striving to move to something better, and I actually deeply enjoy the direction that’s headed by trend.

Bringing the Dev/Sys example back to arrogance, I was actually a pretty arrogant guy a few years back and actively focus on managing it because I tend to be a little more driven than most and get frustrated in the slow lane.  Every once in a while it creeps up and I learn something every time.  I’ve found through careful self observation over the years that usually when I have to fight that the hardest is when I’m around arrogant people.  Most people who come across arrogantly seem to me to be the same way.  I’m still deciding if it is our shared competitive natures or subconscious aggression levels, or if it’s just some defensive mechanism that people unconsciously learn from each other.  If it’s the latter, that’s almost kind of sci-fi, like, a thing that grows and evolves as it gets passed from person to person and matures in interactions — almost like it has a life of its own.

Ah well.  I’m not caught up on Game Of Thrones yet.  Examplar?  Nah.

Maybe next week.

Ivory Tower is a GTD-based task management tool for the Linux Desktop, written in Python using the GTK libraries.

You can download it and use it at its dedicated github repository here.

Home Machine Refresh

or

Full Disk Encryption and Me

Did some neat stuff over the last few weeks and didn’t want people to think I was dead — I’ve just been hella busy.

First, I’ve finished my transition back to Fedora from Ubuntu.  I’d always been a little hesitant about Ubuntu because of the stigma associated with it but I found it to be incredibly well put together and a top notch development environment — and found that moving to Fedora was more a matter of utility than preference due to how dm-crypt / LUKS configuration worked in the installer.  I probably could have pre-worked the partitioning and re-installed ubuntu during the refresh, but opted not to.

Not to say that F25’s encrypted LVM setup was glitch-free.  It was painful and it took hours to get a deviation from the default options to install due to a bug in how pre-existing EFI partitions are [not] detected by it — which was easy compared to Ubuntu’s strange, strange tendency to mount (and fail to unmount) the drives you’re trying to rewrite the partition tables to.  Absolute madness.

Overall, I’m very pleased with the end result.  My machine has a dead man switch for the bootloader and an additional key required to mount anything.  Due to the configuration in use, if I lost the key or deadman switch even a large intelligence agency would likely be unable to recover the data in a usable state.

I’m back in myi3 environment because it’s comfortable to me, but, I am eyeballing awesomeWM.  Not sure yet if I want to make the switch as I’ve just started to explore compositing managers in i3 and have made some neat eye candy for the new setup.

MFG-provided GPU drivers

I didn’t test Fedora’s proprietary graphics driver support because I’m so burned out on that game.

A Super Cool Tool for Reading RFC’s

Found a neat tool that should be, but, alas, is not in the Fedora repos, and I have no idea why after using it.  By bfontaine, it’s aptly titled rfc.

https://github.com/bfontaine/rfc

This is going to be immensely helpful in quoting RFC’s nobody knows I read for fun and profit even though I repeatedly cite them.

And here’s a screenshot of the existing rig:

Obviously quite alot more polish to go as the logo from lightdm is being superimposed onto my background, and I’m not sure why, and it’s annoyingly right on top of my carefully positioned skull and crossbones, which is an absolute necessity for my wallpaper..Not to mention font rendering, while really quite good, is still a little raw for what I like.

I’m using the i3-gaps branch of i3 with i3blocks as a drop-in replacement for i3bar, with (currently) xcompmgr as a compositing manager until I can find one that isn’t exclusively for wayland and also isn’t unstable.

SURRO PROJECTS

As for SURRO projects, I’ve been picking away at Ivory Tower a bit considering a full re-write as a break from Examplar so I don’t burn out.

-CP