Error "Cannot read properties of null (reading 'ActivityLogs')" when trying to deploy - Octopus Cloud

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

刚毅的酱牛肉 · 'DataFrame' object ...· 6 月前 ·

严肃的苹果 · SVG Elements Mouse ...· 7 月前 ·

发呆的骆驼 · Every time I run an ...· 7 月前 ·

傻傻的双杠 · css ...· 7 月前 ·

宽容的野马 · [安卓pc模拟器exagear]2022.9 ...· 10 月前 ·

I’m having lots or problems deploying one of our projects to the TEST environment/server. Other projects are working fine including deploying to this environment. It gets to our “Run migrations” step and then gives me the following message. It also sometimes comes up with “out of memory exception”. Once this happens I have to restart the tentacle to try again but I still come up against errors. Weirdly it has deployed to our DEV environment/server properly.

An unexpected error occurred in Octopus v2021.2.7428: Cannot read properties of null (reading ‘ActivityLogs’)

Hey Darren,

Thanks for posting your issue, and sorry to hear you’re experiencing that behavior.

I was wondering if you see anything noteworthy in the tentacle logs associated with the timing of this error? You can find them under C:\Octopus\Logs or C:\Octopus<tentacleinstance>\Logs if you have multiple tentacles installed.

If there’s nothing there, I may need to log into your instance to review the process and task logs. We have to have specific permission from you to do this, so let me know if that’s ok and I can take a look.

I’m looking forward to hearing back from you.

Best,
Patrick

This looks noteworthy:

ERROR https://redacted.octopus.app:10943/ 8 Halibut client exception: An error occurred when sending a request to ‘ https://redacted.octopus.app:10943/ ’, after the request began: XYZ. Retrying in 120.0 seconds
Halibut.HalibutClientException: An error occurred when sending a request to ‘ https://redacted.octopus.app:10943/ ’, after the request began: XYZ —> System.Security.Authentication.AuthenticationException: XYZ
at Halibut.Transport.Protocol.MessageExchangeStream.ExpectProceeed()
at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsSubscriber(Uri subscriptionId, Func`2 incomingRequestProcessor, Int32 maxAttempts)
at Halibut.Transport.SecureClient.ExecuteTransaction(ExchangeAction protocolHandler, CancellationToken cancellationToken)
— End of inner exception stack trace —
at Halibut.Transport.SecureClient.HandleError(Exception lastError, Boolean retryAllowed)
at Halibut.Transport.SecureClient.ExecuteTransaction(ExchangeAction protocolHandler, CancellationToken cancellationToken)
at Halibut.Transport.PollingClient.ExecutePollingLoop(Object ignored)

I’ll also upload the full logs for your perusal. Feel free to log into our instance until further notice.

OctopusTentacle.0.txt (11.0 KB) OctopusTentacle.1.txt (2.1 KB) OctopusTentacle.2.txt (6.1 KB) OctopusTentacle.3.txt (2.1 KB) OctopusTentacle.4.txt (10.1 KB) OctopusTentacle.5.txt (6.8 KB) OctopusTentacle.6.txt (2.1 KB) OctopusTentacle.txt (7.1 KB)

Hi @darren.woodford !

Thanks for attaching those logs!
I’m just stepping in for Patrick to keep the ball rolling here.

Out of curiosity, is it the same deployment target that the ‘Run migrations’ step is failing on each time?
If this is the case, do you have a different machine available that you can test the deployment of this project to?

Additionally, would you please be able to attach the task log for the deployment that failed with the error message in the OP, please?

Thanks again!

Kind Regards,

How do I get this please? When I click on the release / deployment it gives me this error.

You’re welcome to login to our instance if that will help.

Hi @darren.woodford ,

Apologies, you’re right - I can’t even access the raw task log via API, the same message is being thrown.

I did manage to find the following error message in the server diagnostic logs:

Could not load file or assembly ‘C:\Octopus\Migrations\Kondo\1.19.0.52-revA\Migrations.dll’ or one of its dependencies. The given assembly name or codebase was invalid. (Exception from HRESULT: 0x80131047) System.OutOfMemoryException: Exception of type ‘System.OutOfMemoryException’ was thrown.

Does this error message mean anything to you in particular?
Has this deployment step worked in the past and is now only recently failing?
Have there been any modifications to the deployment process or the .dll you’re using?

Kind Regards,

I’ve cracked it. There was a Run Migrations step that used the Migrations.dll etc that should have been copied across in the previous step. However the previous step was accidentally limited to the Dev environment. Fixed it by making previous step run on all envs. However I think this might need a littel TLC from Octopus (Dev?) side. It should probably handle that a little more gracefully and not completely trash the log so you can’t analyse the problem. Thanks for your help though!

Hi Darren,

Thanks for the update!
We also did some investigating on our side, I noticed you managed to get a successful release out, that’s great!

We managed to pull the log down from the server your instance is running on and it looks like the out of memory exception occurs due to the task log being so huge that it struggles to load, this one was over 1GB and a previous task log on the instance was 3GB+.

The task log seemed to indicate a problem with the Migrator.dll, which then retried over and over again with an increasing number of pipe characters (|), here’s a screenshot of line 50,000 of the log:

It’s rare that errors like these show up but we appreciate your feedback and I’ll mention it internally to see if there is anything we can do to help either prevent the issue or make the issue more clear for troubleshooting.

If you have any other questions or run into any issues again in future, don’t hesitate to reach out! We’re always happy to help how we can.

Kind Regards,

Wowsers! That is huge. Is that our fault or anything we should or could do? I recently migrated this project from onprem so it’s not like there should be a lot of history on it.

If you look at step 4 “Run Migrations” you will see that it uses a fairly trivial “Run Migrations - Fluent Migrator v3” step template. Interestingly I see that the active line of this template ends with | Write-Output as follows.

C:\FluentMigrator\v3\migrate /connection "#{MigrationDatabaseConnectionString}" /db SqlServer /target "C:\Octopus\Migrations\#{Octopus.Project.Name}\#{Octopus.Release.Number}\#{OctopusPathToDll}" --timeout=300 | Write-Output

I’ve inherited this code so I don’t know why it does that. I would have thought the output from this command is written anyway so I don’t see any need to pipe it to Write-Output again…

Do you think this could be the cause of the repeating log?

Hi Darren,

There weren’t many task logs on your server that breached 1gb, I think there were only three. Two of them were releases from this project at least. I wasn’t able to confirm if the third one was also of the same project.

In complete honesty, I’m not sure what causes the looping error message. It could be something related to the commands you’ve linked in your last message or to do with how something within the package behaves.
It seems from the message that the .dll couldn’t be located (which makes sense after the explanation you’ve given).
It could be that something inside the package ‘FluentMigrator’ is logging the error and then retrying continuously but I’m not sure…

A good way to avoid the same thing happening again would be to place a step in-between the data migration and the step that copies the files required that checks to see if the files are in the proper location before proceeding to the next step. This would allow the release to fail with the appropriate message in the strange circumstance that the required prerequisites aren’t where they’re supposed to be.

This does seem to function as more of a workaround but without further testing or looking into the whole process in-depth it’s difficult to see where the issue stems from.

Kind Regards,