技术学院_新闻中心_杭州俍辰科技有限公司

07-11

2017

基于Maven+SSM整合shiro+Redis实现后台管理项目

本项目由卖咸鱼叔叔开发完成，欢迎大神指点，慎重抄袭！参考了sojson提供的demo，和官方文档介绍。完整实现了用户、角色、权限CRUD及分页，还有shiro的登录认证+授权访问控制。项目架构：Maven + SpringMVC + Spring + Mybatis + Shiro + Redis数据库：MySql前端框架：H-ui 首先创建Maven项目1.pom.xml 加入依赖包<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>sys</groupId> <artifactId>sys</artifactId> <packaging>war</packaging> <version>0.0.1-SNAPSHOT</version> <name>sys Maven Webapp</name> <url>http://maven.apache.org</url> <dependencies>   <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context-support</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-orm</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-webmvc</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-aspects</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.6.0</version> </dependency>   <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis</artifactId> <version>3.3.0</version> </dependency>  <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis-spring</artifactId> <version>1.2.3</version> </dependency>  <dependency> <groupId>com.github.pagehelper</groupId> <artifactId>pagehelper</artifactId> <version>4.1.6</version> </dependency>  <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.37</version> </dependency>  <dependency> <groupId>com.mchange</groupId> <artifactId>c3p0</artifactId> <version>0.9.5</version> </dependency>  <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> <version>2.9.0</version> </dependency>  <dependency> <groupId>commons-fileupload</groupId> <artifactId>commons-fileupload</artifactId> <version>1.3.1</version> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.5</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-email</artifactId> <version>1.4</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.7</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.2</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpcore</artifactId> <version>4.4.4</version> </dependency>  <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> </dependency>  <dependency> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> <version>3.1.0</version> <scope>provided</scope> </dependency>  <dependency> <groupId>javax.servlet.jsp</groupId> <artifactId>jsp-api</artifactId> <version>2.2</version> <scope>provided</scope> </dependency>  <dependency> <groupId>org.glassfish.web</groupId> <artifactId>jstl-impl</artifactId> <version>1.2</version> <exclusions> <exclusion> <groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId> </exclusion> <exclusion> <groupId>javax.servlet.jsp</groupId> <artifactId>jsp-api</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>net.sf.json-lib</groupId> <artifactId>json-lib</artifactId> <version>2.4</version> <classifier>jdk15</classifier> </dependency> <dependency> <groupId>commons-httpclient</groupId> <artifactId>commons-httpclient</artifactId> <version>3.1</version> </dependency>  <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-core</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-spring</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-cas</artifactId> <version>1.2.3</version> <exclusions> <exclusion> <groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-web</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-ehcache</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-quartz</artifactId> <version>1.2.3</version> </dependency>  </dependencies> <build> <finalName>sys</finalName> <plugins>  <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> </plugins> </build></project>2.SSM框架配置beans.xml<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:context="http://www.springframework.org/schema/context" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd">  <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"> <property name="driverClass" value="com.mysql.jdbc.Driver"/> <property name="jdbcUrl" value="jdbc:mysql://localhost:3306/sys_test?characterEncoding=UTF8&allowMultiQueries=true"/> <property name="user" value="root"/> <property name="password" value="root"/> <property name="maxPoolSize" value="100"/> <property name="minPoolSize" value="10"/> <property name="maxIdleTime" value="60"/> </bean>  <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean"> <property name="dataSource" ref="dataSource"/> <property name="configLocation" value="classpath:mybatis-config.xml"></property> </bean>  <bean class="org.mybatis.spring.mapper.MapperScannerConfigurer"> <property name="basePackage" value="com.sys.mapper" /> </bean>  <bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="dataSource" /> </bean>  <tx:advice id="txAdvice" transaction-manager="txManager"> <tx:attributes> <tx:method name="select*" propagation="SUPPORTS" read-only="true"/> <tx:method name="page*" propagation="SUPPORTS" read-only="true"/> <tx:method name="is*" propagation="SUPPORTS" read-only="true"/> <tx:method name="*" propagation="REQUIRED" read-only="false"/> </tx:attributes> </tx:advice>  <aop:config> <aop:pointcut id="txPointcut" expression="execution(* com.sys.service..*.*(..))" /> <aop:advisor advice-ref="txAdvice" pointcut-ref="txPointcut" /> </aop:config>   <context:component-scan base-package="com.sys.service"></context:component-scan>  <aop:aspectj-autoproxy/> </beans>dispatcher-servlet.xml<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:context="http://www.springframework.org/schema/context" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd">  <context:component-scan base-package="com.sys" />  <mvc:annotation-driven />  <aop:aspectj-autoproxy/>  <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="prefix" value="/WEB-INF/jsp/"></property> <property name="suffix" value=".jsp"></property> </bean>  <bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver"> <property name="maxUploadSize" value="104857600" /> <property name="defaultEncoding" value="UTF-8" /> <property name="maxInMemorySize" value="40960" /> </bean>  <mvc:resources location="/css/" mapping="/css/**" /> <mvc:resources location="/js/" mapping="/js/**" /> <mvc:resources location="/images/" mapping="/images/**" /> <mvc:resources location="/skin/" mapping="/skin/**" /> <mvc:resources location="/lib/" mapping="/lib/**" /> </beans>mybatis-config.xml<?xml version="1.0" encoding="UTF-8" ?><!DOCTYPE configuration PUBLIC "-//mybatis.org//DTD Config 3.0//EN" "http://mybatis.org/dtd/mybatis-3-config.dtd"><configuration> <settings>  <setting name="logImpl" value="LOG4J2"/> </settings> <typeAliases>  <package name="com.sys.pojo"/> </typeAliases> <mappers>  <package name="com.sys.mapper"/> </mappers></configuration>Log4j2.xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE xml><Configuration status="OFF"> <Appenders> <Console name="CONSOLE" target="SYSTEM_OUT"> <PatternLayout pattern="%d{HH:mm:ss.SSS} %-5level %logger{0} - %msg%n" /> </Console> <RollingFile name="ROLLING" fileName="/logs/ups-manager/log.log" filePattern="/logs/log_%d{yyyy-MM-dd}_%i.log"> <PatternLayout pattern="%d %p %c{1.} [%t] %m%n"/> <Policies> <TimeBasedTriggeringPolicy modulate="true" interval="1"/> <SizeBasedTriggeringPolicy size="1024 KB"/> </Policies> <DefaultRolloverStrategy max="100"/> </RollingFile> </Appenders> <Loggers> <Root level="debug"> <AppenderRef ref="CONSOLE" /> <AppenderRef ref="ROLLING"/> </Root>  <Logger name="org.mybatis.spring" level="error"> <AppenderRef ref="CONSOLE"/> </Logger> </Loggers></Configuration>web.xml<?xml version="1.0" encoding="UTF-8"?><web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd" id="WebApp_ID" version="3.1">  <listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener>  <filter> <filter-name>characterEncodingFilter</filter-name> <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class> <init-param> <param-name>encoding</param-name> <param-value>UTF-8</param-value> </init-param> <init-param>  <param-name>forceEncoding</param-name> <param-value>true</param-value> </init-param> </filter> <filter-mapping> <filter-name>characterEncodingFilter</filter-name>  <url-pattern>/*</url-pattern> </filter-mapping> <context-param> <param-name>contextConfigLocation</param-name> <param-value> classpath:spring-shiro.xml, classpath:beans.xml, classpath:dispatcher-servlet.xml </param-value> </context-param>  <filter> <filter-name>shiroFilter</filter-name> <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class> <init-param> <param-name>targetFilterLifecycle</param-name> <param-value>true</param-value> </init-param> </filter> <filter-mapping> <filter-name>shiroFilter</filter-name> <url-pattern>/*</url-pattern> </filter-mapping>  <servlet> <servlet-name>dispatcher</servlet-name> <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> <init-param> <param-name>contextConfigLocation</param-name> <param-value>classpath:dispatcher-servlet.xml</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>dispatcher</servlet-name> <url-pattern>/</url-pattern> </servlet-mapping> </web-app>3.实现用户、角色、权限页面操作功能，这里就不贴代码了4.接下来就是shiro，登录认证和授权只需要继承AuthorizingRealm，其继承了 AuthenticatingRealm（即身份验证），而且也间接继承了 CachingRealm（带有缓存实现）。身份认证重写doGetAuthenticationInfo方法，授权重写doGetAuthorizationInfo方法。Shiro 默认提供的 Realm 身份认证流程流程如下：首先调用 Subject.login(token) 进行登录，其会自动委托给 Security Manager，调用之前必须通过 SecurityUtils.setSecurityManager() 设置；SecurityManager 负责真正的身份验证逻辑；它会委托给 Authenticator 进行身份验证；Authenticator 才是真正的身份验证者，Shiro API 中核心的身份认证入口点，此处可以自定义插入自己的实现；Authenticator 可能会委托给相应的 AuthenticationStrategy 进行多 Realm 身份验证，默认 ModularRealmAuthenticator 会调用 AuthenticationStrategy 进行多 Realm 身份验证；Authenticator 会把相应的 token 传入 Realm，从 Realm 获取身份验证信息，如果没有返回 / 抛出异常表示身份验证失败了。此处可以配置多个 Realm，将按照相应的顺序及策略进行访问。代码实现：/** * */package com.sys.shiro;import javax.annotation.Resource;import org.apache.shiro.SecurityUtils;import org.apache.shiro.authc.AccountException;import org.apache.shiro.authc.AuthenticationInfo;import org.apache.shiro.authc.AuthenticationToken;import org.apache.shiro.authc.DisabledAccountException;import org.apache.shiro.authc.SimpleAuthenticationInfo;import org.apache.shiro.authz.AuthorizationInfo;import org.apache.shiro.authz.SimpleAuthorizationInfo;import org.apache.shiro.realm.AuthorizingRealm;import org.apache.shiro.subject.PrincipalCollection;import org.apache.shiro.subject.SimplePrincipalCollection;import org.apache.shiro.util.ByteSource;import com.sys.pojo.AdminUser;import com.sys.service.AdminUserService;/** * @ClassName: MyRealm * @Description: shiro 认证 + 授权重写 */public class MyRealm extends AuthorizingRealm { @Resource AdminUserService adminUserService; /* (non-Javadoc) * @see org.apache.shiro.realm.AuthorizingRealm#doGetAuthorizationInfo(org.apache.shiro.subject.PrincipalCollection) */ /** * 授权Realm */ @Override protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principals) { String account = (String)principals.getPrimaryPrincipal(); AdminUser pojo = new AdminUser(); pojo.setAccount(account); Long userId = adminUserService.selectOne(pojo).getId(); SimpleAuthorizationInfo info = new SimpleAuthorizationInfo(); /**根据用户ID查询角色（role），放入到Authorization里.*/ info.setRoles(adminUserService.findRoleByUserId(userId)); /**根据用户ID查询权限（permission），放入到Authorization里.*/ info.setStringPermissions(adminUserService.findPermissionByUserId(userId)); return info; } /* (non-Javadoc) * @see org.apache.shiro.realm.AuthenticatingRealm#doGetAuthenticationInfo(org.apache.shiro.authc.AuthenticationToken) */ /** * 登录认证Realm */ @Override protected AuthenticationInfo doGetAuthenticationInfo(AuthenticationToken token) { String username = (String)token.getPrincipal(); String password = new String((char[])token.getCredentials()); AdminUser user = adminUserService.login(username, password); if(null==user){ throw new AccountException("帐号或密码不正确！"); } if(user.getIsDisabled()){ throw new DisabledAccountException("帐号已经禁止登录！"); } //**密码加盐**交给AuthenticatingRealm使用CredentialsMatcher进行密码匹配 return new SimpleAuthenticationInfo(user.getAccount(),user.getPassword(),ByteSource.Util.bytes("3.14159"), getName()); } /** * 清空当前用户权限信息 */ public void clearCachedAuthorizationInfo() { PrincipalCollection principalCollection = SecurityUtils.getSubject().getPrincipals(); SimplePrincipalCollection principals = new SimplePrincipalCollection( principalCollection, getName()); super.clearCachedAuthorizationInfo(principals); } /** * 指定principalCollection 清除 */ public void clearCachedAuthorizationInfo(PrincipalCollection principalCollection) { SimplePrincipalCollection principals = new SimplePrincipalCollection( principalCollection, getName()); super.clearCachedAuthorizationInfo(principals); } }shiro的配置：spring-shiro.xml<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:util="http://www.springframework.org/schema/util" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd"> <bean id="shiroFilter" class="org.apache.shiro.spring.web.ShiroFilterFactoryBean">  <property name="securityManager" ref="securityManager"></property>  <property name="loginUrl" value="/login"></property>  <property name="unauthorizedUrl" value="/unauthorized.jsp"></property>  <property name="filters"> <map>  <entry key="kickout" value-ref="KickoutSessionControlFilter"/> </map> </property>   <property name="filterChainDefinitions"> <value> /js/**=anon /css/**=anon /images/**=anon /skin/**=anon /lib/**=anon /nodel/**=anon /WEB-INF/jsp/**=anon /adminUserLogin/**=anon /**/submitLogin.do=anon /**=user,kickout </value> </property> </bean>  <bean id="lifecycleBeanPostProcessor" class="org.apache.shiro.spring.LifecycleBeanPostProcessor" />  <bean id="securityManager" class="org.apache.shiro.web.mgt.DefaultWebSecurityManager"> <property name="realm" ref="MyRealm"/> <property name="rememberMeManager" ref="rememberMeManager"/> </bean> <bean id="MyRealm" class="com.sys.shiro.MyRealm" > <property name="cachingEnabled" value="false"/> </bean>  <bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean"> <property name="staticMethod" value="org.apache.shiro.SecurityUtils.setSecurityManager"/> <property name="arguments" ref="securityManager"/> </bean>  <bean id="sessionIdCookie" class="org.apache.shiro.web.servlet.SimpleCookie"> <constructor-arg value="rememberMe"/> <property name="httpOnly" value="true"/> <property name="maxAge" value="-1"/> </bean>  <bean id="rememberMeCookie" class="org.apache.shiro.web.servlet.SimpleCookie"> <constructor-arg value="rememberMe"/> <property name="httpOnly" value="true"/>  <property name="maxAge" value="604800"/> </bean>  <bean id="rememberMeManager" class="org.apache.shiro.web.mgt.CookieRememberMeManager">  <property name="cipherKey" value="#{T(org.apache.shiro.codec.Base64).decode('3AvVhmFLUs0KTA3Kprsdag==')}"/> <property name="cookie" ref="rememberMeCookie"/> </bean>  <bean id="RememberMeFilter" class="com.sys.shiro.RememberMeFilter" />    <bean id="KickoutSessionControlFilter" class="com.sys.shiro.KickoutSessionControlFilter"> </bean> </beans>5.登录即密码失败多次后锁定/** * */package com.sys.controller;import java.util.LinkedHashMap;import java.util.Map;import javax.annotation.Resource;import javax.servlet.http.HttpServletRequest;import org.apache.logging.log4j.LogManager;import org.apache.logging.log4j.Logger;import org.apache.shiro.SecurityUtils;import org.apache.shiro.authc.AccountException;import org.apache.shiro.authc.AuthenticationException;import org.apache.shiro.authc.DisabledAccountException;import org.apache.shiro.authc.ExcessiveAttemptsException;import org.apache.shiro.authc.UsernamePasswordToken;import org.apache.shiro.subject.Subject;import org.springframework.stereotype.Controller;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestMethod;import org.springframework.web.bind.annotation.ResponseBody;import com.sys.pojo.AdminUser;import com.sys.service.AdminUserService;import com.sys.common.JedisUtils;import com.sys.shiro.ShiroUtils;/** * @ClassName: LoginController * @Description: 登录*/@Controllerpublic class LoginController{ protected final static Logger logger = LogManager.getLogger(LoginController.class); protected Map<String, Object> resultMap = new LinkedHashMap<String, Object>(); @Resource AdminUserService adminUserService; /** * @Description: 登录认证 * @param um 登录账号 * @param pw 登录密码 * @param rememberMe 记住我 * @param request * @return * @throws * @author lao * @Date 2018年1月15日下午12:24:19 * @version 1.00 */ @RequestMapping(value="/submitLogin.do",method=RequestMethod.POST) @ResponseBody public Map<String,Object> submitLogin(String um,String pw,boolean rememberMe,HttpServletRequest request){ Subject subject = SecurityUtils.getSubject(); UsernamePasswordToken token = new UsernamePasswordToken(um,ShiroUtils.getStrByMD5(pw)); try{ token.setRememberMe(rememberMe); subject.login(token); JedisUtils.del(um); logger.info("------------------身份认证成功-------------------"); resultMap.put("status", 200); resultMap.put("message", "登录成功！"); } catch (DisabledAccountException dax) { logger.info("用户名为:" + um + " 用户已经被禁用！"); resultMap.put("status", 500); resultMap.put("message", "帐号已被禁用！"); } catch (ExcessiveAttemptsException eae) { logger.info("用户名为:" + um + " 用户登录次数过多，有暴力破解的嫌疑！"); resultMap.put("status", 500); resultMap.put("message", "登录次数过多！"); } catch (AccountException ae) { logger.info("用户名为:" + token.getPrincipal() + " 帐号或密码错误！"); String excessiveInfo = ExcessiveAttemptsInfo(um); if(null!=excessiveInfo){ resultMap.put("status", 500); resultMap.p

01-17

2018

解Bug之路：记一次JVM堆外内存泄露Bug的查找

前言JVM的堆外内存泄露的定位一直是个比较棘手的问题。此次的Bug查找从堆内内存的泄露反推出堆外内存，同时对物理内存的使用做了定量的分析，从而实锤了Bug的源头。笔者将此Bug分析的过程写成博客，以飨读者。由于物理内存定量分析部分用到了linux kernel虚拟内存管理的知识，读者如果有兴趣了解请看ulk3(《深入理解linux内核第三版》)内存泄露Bug现场一个线上稳定运行了三年的系统，从物理机迁移到docker环境后，运行了一段时间，突然被监控系统发出了某些实例不可用的报警。所幸有负载均衡，可以自动下掉节点，如下图所示:登录到对应机器上后，发现由于内存占用太大，触发OOM，然后被linux系统本身给kill了。应急措施紧急在出问题的实例上再次启动应用，启动后，内存占用正常，一切Okay。奇怪现象当前设置的最大堆内存是1792M，如下所示:-Xmx1792m -Xms1792m -Xmn900m -XX:PermSi ze=256m -XX:MaxPermSize=256m -server -Xss512k查看操作系统层面的监控，发现内存占用情况如下图所示:上图蓝色的线表示总的内存使用量，发现一直涨到了4G后，超出了系统限制。很明显，有堆外内存泄露了。查找线索gc日志一般出现内存泄露，笔者立马想到的就是查看当时的gc日志。本身应用所采用框架会定时打印出对应的gc日志，遂查看，发现gc日志一切正常。对应日志如下:查看了当天的所有gc日志，发现内存始终会回落到170M左右，并无明显的增加。要知道JVM进程本身占用的内存可是接近4G(加上其它进程,例如日志进程就已经到4G了)，进一步确认是堆外内存导致。排查代码打开线上服务对应对应代码，查了一圈，发现没有任何地方显式利用堆外内存，其没有依赖任何额外的native方法。关于网络IO的代码也是托管给Tomcat，很明显，作为一个全世界广泛流行的Web服务器，Tomcat不大可能有堆外内存泄露。进一步查找由于在代码层面没有发现堆外内存的痕迹，那就继续找些其它的信息，希望能发现蛛丝马迹。Dump出JVM的Heap堆由于线上出问题的Server已经被kill，还好有其它几台，登上去发现它们也占用了很大的堆外内存，只是还没有到触发OOM的临界点而已。于是就赶紧用jmap dump了两台机器中应用JVM的堆情况，这两台留做现场保留不动，然后将其它机器迅速重启，以防同时被OOM导致服务不可用。使用如下命令dump:jmap -dump:format=b，file=heap.bin [pid]使用MAT分析Heap文件挑了一个heap文件进行分析，堆的使用情况如下图所示:一共用了200多M，和之前gc文件打印出来的170M相差不大，远远没有到4G的程度。不得不说MAT是个非常好用的工具，它可以提示你可能内存泄露的点:这个cachedBnsClient类有12452个实例，占用了整个堆的61.92%。查看了另一个heap文件，发现也是同样的情况。这个地方肯定有内存泄露，但是也占用了130多M，和4G相差甚远。查看对应的代码系统中大部分对于CachedBnsClient的调用，都是通过注解Autowired的，这部分实例数很少。唯一频繁产生此类实例的代码如下所示:@Override public void fun() { BnsClient bnsClient = new CachedBnsClient(); // do something return ; }此CachedBnsClient仅仅在方法体内使用，并没有逃逸到外面，再看此类本身public class CachedBnsClient { private ConcurrentHashMap<String， List<String>> authCache = new ConcurrentHashMap<String， List<String>>(); private ConcurrentHashMap<String， List<URI>> validUriCache = new ConcurrentHashMap<String， List<URI>>(); private ConcurrentHashMap<String， List<URI>> uriCache = new ConcurrentHashMap<String， List<URI>>(); ...... }没有任何static变量，同时也没有往任何全局变量注册自身。换言之，在类的成员(Member)中，是不可能出现内存泄露的。当时只粗略的过了一过成员变量，回过头来细想，还是漏了不少地方的。更多信息由于代码排查下来，感觉这块不应该出现内存泄露(但是事实确是如此的打脸)。这个类也没有显式用到堆外内存，而且只占了130M，和4G比起来微不足道，还是先去追查主要矛盾再说。使用jstack dump线程信息现场信息越多，越能找出蛛丝马迹。先用jstack把线程信息dump下来看下。这一看，立马发现了不同，除了正常的IO线程以及框架本身的一些守护线程外，竟然还多出来了12563多个线程。"Thread-5" daemon prio=10 tid=0x00007fb79426e000 nid=0x7346 waiting on condition [0x00007fb7b5678000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at com.xxxxx.CachedBnsClient$1.run(CachedBnsClient.java:62)而且这些正好是运行再CachedBnsClient的run方法上面！这些特定线程的数量正好是12452个，和cachedBnsClient数量一致!再次check对应代码原来刚才看CachedBnsClient代码的时候遗漏掉了一个关键的点! public CachedBnsClient(BnsClient client) { super(); this.backendClient = client; new Thread() { @Override public void run() { for (; ; ) { refreshCache(); try { Thread.sleep(60 * 1000); } catch (InterruptedException e) { logger.error("出错"， e); } } } }这段代码是CachedBnsClient的构造函数，其在里面创建了一个无限循环的线程，每隔60s启动一次刷新一下里面的缓存!找到关键点在看到12452个等待在CachedBnsClient.run的业务的一瞬间笔者就意识到，肯定是这边的线程导致对外内存泄露了。下面就是根据线程大小计算其泄露内存量是不是确实能够引起OOM了。发现内存计算对不上由于我们这边设置的Xss是512K，即一个线程栈大小是512K，而由于线程共享其它MM单元(线程本地内存是是现在线程栈上的)，所以实际线程堆外内存占用数量也是512K。进行如下计算:12563 * 512K = 6331M = 6.3G整个环境一共4G，加上JVM堆内存1.8G(1792M)，已经明显的超过了4G。(6.3G + 1.8G)=8.1G > 4G如果按照此计算，应用应用早就被OOM了。怎么回事呢？为了解决这个问题，笔者又思考了好久。如下所示:Java线程底层实现JVM的线程在linux上底层是调用NPTL(Native Posix Thread Library)来创建的，一个JVM线程就对应linux的lwp(轻量级进程，也是进程，只不过共享了mm_struct，用来实现线程)，一个thread.start就相当于do_fork了一把。其中，我们在JVM启动时候设置了-Xss=512K(即线程栈大小)，这512K中然后有8K是必须使用的，这8K是由进程的内核栈和thread_info公用的，放在两块连续的物理页框上。如下图所示:众所周知，一个进程(包括lwp)包括内核栈和用户栈，内核栈+thread_info用了8K，那么用户态的栈可用内存就是:512K-8K=504K如下图所示:Linux实际物理内存映射事实上linux对物理内存的使用非常的抠门，一开始只是分配了虚拟内存的线性区，并没有分配实际的物理内存，只有推到最后使用的时候才分配具体的物理内存，即所谓的请求调页。如下图所示:查看smaps进程内存使用信息使用如下命令，查看cat /proc/[pid]/smaps > smaps.txt实际物理内存使用信息，如下所示:7fa69a6d1000-7fa69a74f000 rwxp 00000000 00:00 0 Size: 504 kB Rss: 92 kB Pss: 92 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 92 kB Referenced: 92 kB Anonymous: 92 kB AnonHugePages: 0 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB 7fa69a7d3000-7fa69a851000 rwxp 00000000 00:00 0 Size: 504 kB Rss: 152 kB Pss: 152 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 152 kB Referenced: 152 kB Anonymous: 152 kB AnonHugePages: 0 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB搜索下504KB，正好是12563个，对了12563个线程，其中Rss表示实际物理内存(含共享库)92KB，Pss表示实际物理内存(按比例共享库)92KB(由于没有共享库，所以Rss==Pss)，以第一个7fa69a6d1000-7fa69a74f000线性区来看，其映射了92KB的空间，第二个映射了152KB的空间。如下图所示:挑出符合条件（即size是504K）的几十组看了下，基本都在92K-152K之间，再加上内核栈8K(92+152)/2+8K=130K，由于是估算，取整为128K，即反映此应用平均线程栈大小。注意，实际内存有波动的原因是由于环境不同，从而走了不同的分支，导致栈上的增长不同。重新进行内存计算JVM一开始申请了-Xmx1792m -Xms1792m即1.8G的堆内内存，这里是即时分配，一开始就用物理页框填充。12563个线程，每个线程栈平均大小128K，即:128K * 12563=1570M=1.5G的对外内存取个整数128K，就能反映出平均水平。再拿这个128K * 12563 =1570M = 1.5G，加上JVM的1.8G，就已经达到了3.3G，再加上kernel和日志传输进程等使用的内存数量，确实已经接近了4G，这样内存就对应上了！(注:用于定量内存计算的环境是一台内存用量将近4G，但还没OOM的机器)为什么在物理机上没有应用Down机笔者登录了原来物理机，应用还在跑，发现其同样有堆外内存泄露的现象，其物理内存使用已经达到了5个多G!幸好物理机内存很大，而且此应用发布还比较频繁，所以没有被OOM。Dump了物理机上应用的线程，一共有28737个线程，其中28626个线程等待在CachedBnsClient上。同样用smaps查看进程实际内存信息，其平均大小依旧为128K，因为是同一应用的原因继续进行物理内存计算1.8+(28737 * 128k)/1024K =(3.6+1.8)=5.4G进一步验证了我们的推理。这么多线程应用为什么没有卡顿因为基本所有的线程都睡眠在 Thread.sleep(60 * 1000);//一次睡眠60s上。所以仅仅占用了内存，实际占用的CPU时间很少。总结查找Bug的时候，现场信息越多越好，同时定位Bug必须要有实质性的证据。例如内存泄露就要用你推测出的模型进行定量分析。在定量和实际对不上的时候，深挖下去，你会发现不一样的风景!