Easy GoLang AWS SDK V2 Upgrade

In my day job I have a pretty extensive library of routines I use to wrap cloud automation using the vendor libraries as composed functions. That makes things like deploying a stack, or creating an AMI a bit easier.

Composed functions allow me to build things in the cloud in a consistent way.  I simply call the one method (like awshandler.CreateStack) and it executes all the steps to ensure the right things get done.

I started updating my codebase from aws-sdk to aws-sdk-v2. AWS does have a pretty solid Migration Guide on the AWS SDK for Go V2 documentation web page. Even with this, there were a LOT of things I needed to update.

Imports

The first obvious change was to do a global search and replace for aws/aws-sdk-go/ with aws/aws-sdk-go-v2. That got me a bit of the way through, and exposed a lot of errors in need of fixing.

A few things got moved around, and a few imports were just broken because of that (more on this later).

Session

In my code, I take care of getting a session which involves setting the config and getting a *session.Session to use for connecting to the account. The V2 library deprecated the *session.Session altogether, replacing it with *aws.Config

At first this puzzled me as i had code like sess, err := session.NewSession(myConfig) wherever I needed to get a session. I would then pass the session to subsequent method to use.

For the most part, I just had to remove the import for session, and do another global replace of *session.Session with *aws.Config.

After that, fix the calls where I was doing getting the service to interact with with another global replace of New(sess) with NewFromConfig(*sess)  

For hygiene I also need to go back and change the sess to something like currentConfig, but with this the bits where I manage sessions is fixed.

Types instead of variables

All of the types referenced by the library, got refactored as types libraries. Most references to those types were changed to the actual types instead of pointers.

That means when you have a reference like *ec2.Region in the new library it would be types.Region. In addition, the service library that was previously referenced with *ec2.EC2 is now replaced by *ec2.Client for all the different objects.

Since my libraries are generally dealing with only one type (like EC2 in the above example) I could just change the EC2 to be Client (or whatever object I was dealing with).

The types replacement in the IDE was easy this way too. The IDE is smart enough to know that when I type types.Region that I really am referring to github.com/aws/aws-sdk-go-v2/ec2/types 

Eventually I ran into a file where I was composing multiple objects. I had to change the reference so that the compiler would know which type I was referring to like:

	orgtypes "github.com/aws/aws-sdk-go-v2/service/organizations/types"
	"github.com/aws/aws-sdk-go-v2/service/route53"
	route53types "github.com/aws/aws-sdk-go-v2/service/route53/types"

In hindsight, referring to those types in this way probably would make general sense, so that it would be clear that types.Account refers to an organizations account because the reference would be organizationtypes.Account instead of just a type.

Not everything is a pointer

Next improvement in the V2 library is that not everything is a pointer. In the prior release everything was made into a pointer. There were hacks like using aws.String("some string") when passing a string in a struct (that was also always a pointer). Things were getting passed around as pointers to structs that were filled with more pointers. Great for consistency, not so great for actual coding.

Also all the method calls now require a context in addition to the struct, so for the most part I just added context.Background() to the arguments on the call. In some cases the change to add that and having changed the type declaration was enough. I definitely had to look at each call as I made these changes.

Because of the pointer and type issue, I often had goofy loops to convert things into pointers. Those definitely were hacky and had potential for weird bugs. For example, when passing in parameters to a CloudFormation stack create/update, there was an array of pointers ([]*cloudformation.Parameter) which I typically passed in as a collection of parameters or strings. I would end up having hacky code like:

	var parameters []*cloudformation.Parameter

	for _, parm := range input.Parameters {
		// Since we want pointers, we have to reallocate
		// the parameter before assigining it ...
		var myParameter = parm
		parameters = append(parameters, &myParameter)
	}
	var tags []*cloudformation.Tag

	for _, tag := range input.Tags {
		var myTag = tag
		tags = append(tags, &myTag)
	}

Then remember to pass the parameters as the value for Parameters – with the update, the signature now expected an slice []types.Parameter so because I’d replaced *cloudformation. with types. I could just use the original input.Parameters in the call.

Error handling

Turns out error handling also changed in the V2 library, the old library had awserr that contained most of the error codes that are returned by the API. That definitely made handling each one differently with a case statement pretty easy.

In theory do something different with a recoverable error than with one that you can’t patch and retry.

For example in this one from updating a stack, where a ValidationError is thrown when there are no updates to be performed (which for my purposes is not an error, where anything else would be)

if aerr, ok := updateStackOutput.Error.(awserr.Error); ok {
	switch aerr.Code() {
	case "ValidationError":
		if strings.Contains(aerr.Error(), "No updates are to be performed") {
			return
		}
		log.Printf("%v - UpdateStack("+input.StackName+") code: %v error: %v\n", input.AccountId, aerr.Code(), aerr.Error())

	default:
		log.Printf("%v - UpdateStack("+input.StackName+") code: %v error: %v\n", input.AccountId, aerr.Code(), aerr.Error())
	}
} else {
	// Print the error, cast Error to awserr.Error to get the Code and
	// Message from an error.
	log.Printf("%v - UpdateStack("+input.StackName+") General error: %v\n", input.AccountId, updateStackOutput.Error.Error())
}

Instead of awserr the error is returned as a *smithy.OperationError which provides details about the operation and the failure so I could still check the error for recoverable errors like above:

	if updateStackOutput.Error != nil {
		var oe *smithy.OperationError
		if errors.As(err, &oe) {
			if strings.Contains(oe.Error(), "No updates are to be performed") {
				return
			}
			log.Printf("failed to call service: %s, operation: %s, error: %v", oe.Service(), oe.Operation(), oe.Unwrap())
		}
	}

Summary

I had about 5 years of accumulated code. Completing the update of aws-sdk to aws-sdk-v2 took a day or two.

In one or two use cases where the new library didn’t support some older things. For instance SimpleDB. I simply left using the older methods and probably will deprecate as they’re not all that useful.

I did do some similar work for the GCP libraries. That wasn’t quite as straightforward as the AWS one was. Although, I may go back and look at that later. GCP has a multiple versions and each one appears to work very differently.  GCO deprecates methods more often, and leave figuring out the difference on how to migrate to the developer

AWS does a great job with making the API consistent across all the resources. Which means that if you’ve got an example of how to use one, it’s pretty easy to use any other.