--- title: "Discussion on \"How I Investigate a Suspicious Login on a Linux Server\" | Hashnode" description: "Discussion on \"How I Investigate a Suspicious Login on a Linux Server\". Learn step-by-step forensic triage for suspicious Linux logins. Discover how to inspect /proc, trace auth.log, catch reverse shells" url: "https://hashnode.com/posts/how-i-investigate-a-suspicious-login-on-a-linux-server/6ab2a68925a7eb6282aeca0d" --- [Feed](https://hashnode.com/feed) / Discussion # [How I Investigate a Suspicious Login on a Linux Server](https://asepsayyad007.hashnode.dev/how-i-investigate-a-suspicious-login-on-a-linux-server?utm_source=hashnode&utm_medium=feed) Learn step-by-step forensic triage for suspicious Linux logins. Discover how to inspect /proc, trace auth.log, catch reverse shells Sep 22 · 16 min read · asepsayyad007.hashnode.dev Read the article - [#linux](https://hashnode.com/tag/linux) - [#ubuntu](https://hashnode.com/tag/ubuntu) - [#devops](https://hashnode.com/tag/devops) - [#cli](https://hashnode.com/tag/cli) - [#tutorial](https://hashnode.com/tag/tutorial) ## Written by [Asep Sayyad](https://hashnode.com/@asepsayyad007) @asepsayyad007 · Aurangabad [View profile](https://hashnode.com/@asepsayyad007) Linux • DevOps • Self-Hosting • Open Source Linux • DevOps • Open Source | Building AiroDrop—an Apple-like ecosystem for iPhone users on Windows. Sharing real-world projects, automation, and engineering lessons. ## Responses 5 Markdown supported · ⌘↵ to post [Mihai\_LeanZero](https://hashnode.com/@mihaileanzero) [Sep 22](https://hashnode.com/posts/how-i-investigate-a-suspicious-login-on-a-linux-server/6ab2a68925a7eb6282aeca0d/comment/6ab2b1502605f37864874853) This is one of the most thorough triage playbooks I've read here - the ctime-vs-mtime timestomping point and catching the pty-less sshd session via ss instead of w are both things most writeups skip entirely. One gap worth flagging for anyone running this against a cloud-hosted replica specifically: none of the persistence vectors here check the instance metadata service. If db-prod-01 is EC2, an attacker with even a few seconds of shell access can curl 169.254.169.254/latest/meta-data/iam/security-credentials/ and walk off with temporary IAM credentials that keep working long after you've frozen the box and rebuilt from known-good IaC - no local persistence needed at all. Worth checking VPC flow logs or CloudTrail for that request alongside the /proc and ld.so.preload checks (GuardDuty flags it directly as InstanceCredentialExfiltration), since the "rebuild the instance" advice at the end doesn't help if the stolen credentials are still valid off-box. [Asep Sayyad](https://hashnode.com/@asepsayyad007) [Oct 2](https://hashnode.com/posts/how-i-investigate-a-suspicious-login-on-a-linux-server/6ab2a68925a7eb6282aeca0d/comment/6ab2b1502605f37864874853#reply-6abfd6c4ec620802a928efaa) Great point. I focused mainly on host-level persistence in this playbook, but cloud credentials are an important part of the investigation too. Checking the instance metadata access and CloudTrail/GuardDuty activity should definitely be added for EC2 cases. Thanks for pointing this out! [Furqan Ashraf](https://hashnode.com/@jfreaks) [Sep 23](https://hashnode.com/posts/how-i-investigate-a-suspicious-login-on-a-linux-server/6ab2a68925a7eb6282aeca0d/comment/6ab3ed7385d6402093317ce1) Solid playbook, the ctime vs mtime timestomping point alone is worth bookmarking. One thing I'd add right at the top, before any of the /proc work: the opening scenario mentions an SSH session "from an IP address block in a country where your engineering team has zero presence," but that judgment call is doing a lot of work and it's worth making it explicit rather than trusting the alert. A quick geolocation + ASN lookup on the source IP tells you fast whether you're looking at a residential ISP in an unexpected country (bad sign) or just a known corporate VPN exit / cloud NAT range that happens to geolocate oddly (false alarm). ipgeolocation.io's free tier is enough for this, one call gives you country, ASN and organization/network type, so you're not burning time on /proc forensics for what turns out to be a coworker on a company VPN. Worth wiring straight into the alert itself so the country/ASN shows up before anyone even opens a terminal. [Asep Sayyad](https://hashnode.com/@asepsayyad007) [Oct 2](https://hashnode.com/posts/how-i-investigate-a-suspicious-login-on-a-linux-server/6ab2a68925a7eb6282aeca0d/comment/6ab3ed7385d6402093317ce1#reply-6abfd6d0ec620802a928efd3) That's a good point. Country alone shouldn't be treated as proof of a suspicious login. Checking the IP's ASN and organization first can help separate a real anomaly from a VPN or cloud network. I'll definitely consider adding this to the initial triage steps. Thanks! [Furqan Ashraf](https://hashnode.com/@jfreaks) [Oct 5](https://hashnode.com/posts/how-i-investigate-a-suspicious-login-on-a-linux-server/6ab2a68925a7eb6282aeca0d/comment/6ab3ed7385d6402093317ce1#reply-6ac346f803acadaf838233e0) Thanks Asep. One more cheap triage step to put next to the ASN check: run the source IP against a threat feed before anyone opens a terminal. If it is already listed as an attacker or abuse source, you escalate faster. Disclosure, I work at WhoisFreaks, and our threat intelligence feed covers IP threats along with phishing and malware indicators, scored so you can load them into a SIEM or MISP. There are free samples to test with, and the full feed is a paid license: [whoisfreaks.com/products/threat-intelligence-feed](https://whoisfreaks.com/products/threat-intelligence-feed) Any reputation list works for this step. Just treat a clean result as "no known history", not "safe".