CrowdStrike

CrowdStrike

Regarding Windows system failures (BSOD) caused by CrowdStrike Holdings、Inc. security software

We have been informed CrowdStrike Holdings、Inc. (hereinafter CS) that the worldwide system failure on Windows PCs that occurred on July 19th was caused by an update to their security software, Falcon Sensor. As a domestic distributor of CS, Macnica will report any information regarding this failure on our website as soon as we receive it.

We have already received a report from CS that they have identified the cause of the problem and are doing everything they can to restore the service and provide support to their customers. We are also currently doing everything we can to support our contracted customers so that they can restore their business. We ask our customers and sales partners to please refer to the information below and contact us if you have any questions. We appreciate your understanding and cooperation.


Summary

- Windows host crashes (BSOD) related to the Falcon Sensor have been confirmed.

・Technical details of this issue (CS blog post)

・CS's Falcon platform system is operating normally, so if your system is operating normally, even if the Falcon Sensor is installed, there will be no impact on the system's protection. Falcon Complete and Overwatch services will not be interrupted by this incident.

Technical Overview

CS identified the trigger for this issue to be Windows Sensor related content deployment and reverted these changes. The content in question was the channel file located in the %WINDIR%\System32\drivers\CrowdStrike directory.

・If the timestamp of the causative channel file "C-00000291*.sys" is between 13:09 and 14:27 Japan time on July 19, 2024, you may be affected. (If the channel file is after 14:27 Japan time on July 19, 2024, it has been corrected and no recovery action is required.)
Note: It is normal for there to be multiple C-00000291*.sys files in the CrowdStrike directory. If any of the files in the folder have a timestamp later than 2024/7/19 14:27 Japan time, the file will be considered active.

Unaffected devices

・Windows devices that went online after 14:27 on July 19, 2024 (Japan time)

・日本時間2024/7/19 14:27 以降にインストールおよびプロビジョニングされたWindows端末

- Mac or Linux device

Step 1: Identify affected devices

How to identify devices using Advanced Event Search

Instructions for running queries, along with examples, are provided at the end of this page.
*Note: An Insight subscription is required to use this feature.

How to identify a device using the dashboard

ダッシュボードは、影響を受けたチャンネルとCID、および影響を受けたセンサーを表示します。 サブスクリプションによっては、以下のコンソールメニューのいずれかで利用できます。
※注意: ご利用にはInsightのサブスクリプションが必要になります。

 ・次世代 SIEM > ダッシュボード (Next-Gen SIEM > Log management > Dashboard)
 ・調査 > ダッシュボード (Investigate > Dashboards)
 ・ダッシュボード名: hosts_possibly_impacted_by_windows_crashes_granular_status

  ※注意:ダッシュボードはLiveボタンとは併用できません


ダッシュボードの操作方法に関する資料を弊社サポート記事に掲載しておりますので、あわせて参照ください。

Step 2: Recovery

If your host continues to crash and no fixes are being applied from the cloud, you can recover by following the steps below:
Although there are procedures other than those listed below that are published on external websites, we ask that you please follow the procedures published by our company and CS Inc.

Recovery measures for individual devices

・端末を再起動すると、修正されたチャンネルファイルをダウンロードできるようになります。高速な通信を確保するため有線インターネット接続で端末を再起動することをおすすめいたします。

・リブート後にクラッシュが継続する場合
 ・オプション1- 手動削除手順
  こちらのマイクロソフトの記事から復旧のステップをご確認いただけます。

 ・オプション2 – リカバリツールによる削除手順
  リカバリツールは、こちらのKB記事をご参照ください。

  ※注意: 上記のオプションは共にBitLockerで暗号化された端末は回復キーが必要な場合があります

BitLocker Recovery in a Microsoft Environment

 BitLocker Recovery in Microsoft Azure
 ​ ​BitLocker Recovery in a Microsoft Environment using SCCM
 BitLocker Recovery in a Microsoft Environment using Active Directory and GPO​ ​ 
 BitLocker recovery in a Microsoft environment using Ivanti Endpoint Manager​ ​ 
 BitLocker Recovery in Microsoft Environments using ManageEngine Desktop Central​ ​ 
 BitLocker recovery in Microsoft environments using IBM BigFix

BitLocker recovery without recovery key

 BitLocker recovery without recovery key

BitLocker Recovery in the Workspace ONE Portal

 User Access to Recovery Key in the Workspace ONE Portal

BitLocker Recovery with Tanium

 Reference: Windows encryption management

Bitlocker Recovery with Citrix

 BitLocker recovery key

AWS Recovery

 How do I recover AWS resources that were affected by the CrowdStrike Falcon agent?

Azure Recovery

 Azure status

Google Cloud Platform (GCP) Recovery

 Automated Recovery from Blue Screen on Windows Instances in GCP

Recovery procedures for public cloud or similar environments, including virtual

[Option 1]
1. Detach the operating system disk volume from the affected virtual server
2. Create a snapshot or backup of your disk volume before proceeding (precaution against unintentional changes).
3. Attach/mount the volume to the new virtual server
4. Go to the “%WINDIR%\\System32\drivers\CrowdStrike” directory
5. Delete the files that start with “C-00000291” and end with “.sys” (“C-00000291*.sys”)
6. Detach the volume from the new virtual server
7. Attach/mount the fixed volume to the affected virtual server

[Option 2]

Rollback to a snapshot before 13:09, 19 July 2024 (UTC)

Intel vPro Technology Repair Guide

 Remediate CrowdStrike Falcon® update issue on Windows systems with Intel vPro® technology

Rubrikリカバリ

 CrowdStrike & Rubrik Customer Content Update Recovery For Windows Hosts

Cohesity Support

 Cohesity’s support for CrowdStrike’s Falcon Sensor updates

Contact information

Regarding this issue, we have published the following article on our support site and an article from CS Inc. Please refer to the article on our support site, which will be updated from time to time along with this page.
*Our support site also provides information on how to operate the dashboard and case studies related to this issue.

Our support article: https://support.mnc.macnica.co.jp/hc/ja/articles/35269019637657 
CS article: https://supportportal.crowdstrike.com/s/article/Tech-Alert-Windows-crashes-related-to-Falcon-Sensor-2024-07-19

Macnica
CrowdStrike Product Support: crowdstrike@macnica.co.jp
Phone number: 050-3134-3642

Revision history

2024/07/19 21:00 First edition published

2024/07/19 23:00 Updated the following items
・Addition of details
-Addition of current actions (workarounds)
Workaround steps for individual hosts
Workarounds for public cloud or similar environments
-Add link information
・AWS article on workarounds for virtual machines on AWS
・CS article on Bitlocker recovery procedures

2024/07/20 2:45 Updated the following items
Update details
・The following text has been deleted:
  Windows 7 and Windows Server 2008 R2 hosts are not affected by this issue.
-Update current action (workaround)
-Added information about queries
Use Advanced Event Search to query to see affected hosts
-Add link information
- User access recovery key in Workspace ONE portal
-CrowdStrike Holdings、Inc. article on Bitlocker recovery procedures

2024/07/20 7:30
・記事タイトル、概要を更新

2024/07/20 10:30 
・クエリに関する情報を更新
・リンク情報の追記
 ・GCPでのWindowsインスタンスの自動リカバリ
 ・Taniumを利用したWindows暗号化管理
 ・Citrixを利用したBitlockerリカバリ
・リンク情報を更新
 ・AWS上の仮想マシンでの回避手順についてのAWS記事

2024/07/20 12:30
・影響を受けるホストを確認するためのダッシュボードの追加
・個々のホストに対する回避策の手順を更新

2024/07/20 19:00
Bitlockerリカバリキーが利用できない場合の対処方法に関するメーカ記事(Bitlocker recovery without recovery keys)を追記

2024/07/20 21:00
・GCPでのWindowsインスタンスの自動リカバリの記事、Bitlockerのリカバリ手順についてのCS社記事のリンクを更新

2024/07/21 11:00
・Technical Overview、影響を受けない端末を追記
・影響を受ける端末の特定、復旧手順について整理

2024/07/21 20:30
・リカバリツールによる削除手順を追記

2024/07/22 19:15
・影響を受けない端末 の内容を追記
・ステップ1:影響を受ける端末の特定 の内容を更新
・Advanced Event Search(高度なイベント検索) による端末特定方法 を更新

How to identify devices using Advanced Event Search

*Note: An Insight subscription is required to use this feature.

こちらのKB記事をご参照ください

Inquiry/Document request

In charge of Macnica CrowdStrike Co., Ltd.

Mon-Fri 8:45-17:30

日本語
English

This page uses machine translation.


The English version of this site is machine-translated, so some of the expressions may be incorrect. We appreciate your understanding in advance.

このサイトは機械翻訳によって作成されているため、一部表現に誤りがある場合がございます。 予めご了承いただけますと幸いです。