GuavaCache学习笔记三:底层源码阅读

2023-06-07,,

申明:转载自 https://www.cnblogs.com/dennyzhangdd/p/8981982.html

感谢原博主的分享,看到这个写的真好,直接转载来,学习了。

另外也推荐另外一篇GuavaCache的文章:https://ketao1989.github.io/2014/12/19/Guava-Cache-Guide-And-Implement-Analyse/#top


正文

回到顶部

一、引子

缓存有很多种解决方案,常见的是:

1.存储在内存中 : 内存缓存顾名思义直接存储在JVM内存中,JVM宕机那么内存丢失,读写速度快,但受内存大小的限制,且有丢失数据风险。

2.存储在磁盘中: 即从内存落地并序列化写入磁盘的缓存,持久化在磁盘,读写需要IO效率低,但是安全。

3.内存+磁盘组合方式:这种组合模式有很多成熟缓存组件,也是高效且安全的策略,比如redis。

本文分析常用的内存缓存:google cache。源码包:com.google.guava:guava:22.0 jar包下的pcom.google.common.cache包,适用于高并发读写场景,可自定义缓存失效策略。

回到顶部

二、使用方法

2.1 CacheBuilder有3种失效重载模式

1.expireAfterWrite

当 创建 或 写之后的 固定 有效期到达时,数据会被自动从缓存中移除,源码注释如下:

 1   /**指明每个数据实体:当 创建 或 最新一次更新 之后的 固定值的 有效期到达时,数据会被自动从缓存中移除
2 * Specifies that each entry should be automatically removed from the cache once a fixed duration
3 * has elapsed after the entry's creation, or the most recent replacement of its value.
4 *当间隔被设置为0时,maximumSize设置为0,忽略其它容量和权重的设置。这使得测试时 临时性地 禁用缓存且不用改代码。
5 * <p>When {@code duration} is zero, this method hands off to {@link #maximumSize(long)
6 * maximumSize}{@code (0)}, ignoring any otherwise-specified maximum size or weight. This can be
7 * useful in testing, or to disable caching temporarily without a code change.
8 *过期的数据实体可能会被Cache.size统计到,但不能进行读写,数据过期后会被清除。
9 * <p>Expired entries may be counted in {@link Cache#size}, but will never be visible to read or
10 * write operations. Expired entries are cleaned up as part of the routine maintenance described
11 * in the class javadoc.
12 *
13 * @param duration the length of time after an entry is created that it should be automatically
14 * removed
15 * @param unit the unit that {@code duration} is expressed in
16 * @return this {@code CacheBuilder} instance (for chaining)
17 * @throws IllegalArgumentException if {@code duration} is negative
18 * @throws IllegalStateException if the time to live or time to idle was already set
19 */
20 public CacheBuilder<K, V> expireAfterWrite(long duration, TimeUnit unit) {
21 checkState(
22 expireAfterWriteNanos == UNSET_INT,
23 "expireAfterWrite was already set to %s ns",
24 expireAfterWriteNanos);
25 checkArgument(duration >= 0, "duration cannot be negative: %s %s", duration, unit);
26 this.expireAfterWriteNanos = unit.toNanos(duration);
27 return this;
28 }

2.expireAfterAccess

指明每个数据实体:当 创建 或 写 或 读 之后的 固定值的有效期到达时,数据会被自动从缓存中移除。读写操作都会重置访问时间,但asMap方法不会。源码注释如下:

 1 /**指明每个数据实体:当 创建 或 更新 或 访问 之后的 固定值的有效期到达时,数据会被自动从缓存中移除。读写操作都会重置访问时间,但asMap方法不会。
2 * Specifies that each entry should be automatically removed from the cache once a fixed duration
3 * has elapsed after the entry's creation, the most recent replacement of its value, or its last
4 * access. Access time is reset by all cache read and write operations (including
5 * {@code Cache.asMap().get(Object)} and {@code Cache.asMap().put(K, V)}), but not by operations
6 * on the collection-views of {@link Cache#asMap}.
7 * 后面的同expireAfterWrite
8 * <p>When {@code duration} is zero, this method hands off to {@link #maximumSize(long)
9 * maximumSize}{@code (0)}, ignoring any otherwise-specified maximum size or weight. This can be
10 * useful in testing, or to disable caching temporarily without a code change.
11 *
12 * <p>Expired entries may be counted in {@link Cache#size}, but will never be visible to read or
13 * write operations. Expired entries are cleaned up as part of the routine maintenance described
14 * in the class javadoc.
15 *
16 * @param duration the length of time after an entry is last accessed that it should be
17 * automatically removed
18 * @param unit the unit that {@code duration} is expressed in
19 * @return this {@code CacheBuilder} instance (for chaining)
20 * @throws IllegalArgumentException if {@code duration} is negative
21 * @throws IllegalStateException if the time to idle or time to live was already set
22 */
23 public CacheBuilder<K, V> expireAfterAccess(long duration, TimeUnit unit) {
24 checkState(
25 expireAfterAccessNanos == UNSET_INT,
26 "expireAfterAccess was already set to %s ns",
27 expireAfterAccessNanos);
28 checkArgument(duration >= 0, "duration cannot be negative: %s %s", duration, unit);
29 this.expireAfterAccessNanos = unit.toNanos(duration);
30 return this;
31 }

3.refreshAfterWrite

指明每个数据实体:当 创建 或 写 之后的 固定值的有效期到达时,数据会被自动刷新(注意不是删除是异步刷新,不会阻塞读取,先返回旧值,异步重载到数据返回后复写新值)。源码注释如下:

 1 /**指明每个数据实体:当 创建 或 更新 之后的 固定值的有效期到达时,数据会被自动刷新。刷新方法在LoadingCache接口的refresh()申明,实际最终调用的是CacheLoader的reload()
2 * Specifies that active entries are eligible for automatic refresh once a fixed duration has
3 * elapsed after the entry's creation, or the most recent replacement of its value. The semantics
4 * of refreshes are specified in {@link LoadingCache#refresh}, and are performed by calling
5 * {@link CacheLoader#reload}.
6 * 默认reload是同步方法,所以建议用户覆盖reload方法,否则刷新将在无关的读写操作间操作。
7 * <p>As the default implementation of {@link CacheLoader#reload} is synchronous, it is
8 * recommended that users of this method override {@link CacheLoader#reload} with an asynchronous
9 * implementation; otherwise refreshes will be performed during unrelated cache read and write
10 * operations.
11 *
12 * <p>Currently automatic refreshes are performed when the first stale request for an entry
13 * occurs. The request triggering refresh will make a blocking call to {@link CacheLoader#reload}
14 * and immediately return the new value if the returned future is complete, and the old value
15 * otherwise.触发刷新操作的请求会阻塞调用reload方法并且当返回的Future完成时立即返回新值,否则返回旧值。
16 *
17 * <p><b>Note:</b> <i>all exceptions thrown during refresh will be logged and then swallowed</i>.
18 *
19 * @param duration the length of time after an entry is created that it should be considered
20 * stale, and thus eligible for refresh
21 * @param unit the unit that {@code duration} is expressed in
22 * @return this {@code CacheBuilder} instance (for chaining)
23 * @throws IllegalArgumentException if {@code duration} is negative
24 * @throws IllegalStateException if the refresh interval was already set
25 * @since 11.0
26 */
27 @GwtIncompatible // To be supported (synchronously).
28 public CacheBuilder<K, V> refreshAfterWrite(long duration, TimeUnit unit) {
29 checkNotNull(unit);
30 checkState(refreshNanos == UNSET_INT, "refresh was already set to %s ns", refreshNanos);
31 checkArgument(duration > 0, "duration must be positive: %s %s", duration, unit);
32 this.refreshNanos = unit.toNanos(duration);
33 return this;
34 }

2.2 测试验证

1)定义一个静态的LoadingCache,用cacheBuilder构造缓存,分别定义了同步load(耗时2秒)和异步reload(耗时2秒)方法。

2)在main方法中,往缓存中设置值,定义3个线程,用CountDownLatch倒计时器模拟3个线程并发读取缓存,最后在主线程分别5秒、0.5秒、2秒时get缓存。

测试代码如下:

  1 package guava;
2
3 import com.google.common.cache.CacheBuilder;
4 import com.google.common.cache.CacheLoader;
5 import com.google.common.cache.LoadingCache;
6 import com.google.common.util.concurrent.ListenableFuture;
7 import com.google.common.util.concurrent.ListeningExecutorService;
8 import com.google.common.util.concurrent.MoreExecutors;
9
10 import java.util.Date;
11 import java.util.Random;
12 import java.util.concurrent.Callable;
13 import java.util.concurrent.CountDownLatch;
14 import java.util.concurrent.Executors;
15 import java.util.concurrent.TimeUnit;
16
17 /**
18 * @ClassName guava.LoadingCacheTest
19 * @Description 注意refresh并不会主动刷新,而是被检索触发更新value,且随时可返回旧值
20 * @Author denny
21 * @Date 2018/4/28 下午12:10
22 */
23 public class LoadingCacheTest {
24
25 // guava线程池,用来产生ListenableFuture
26 private static ListeningExecutorService service = MoreExecutors.listeningDecorator(
27 Executors.newFixedThreadPool(10));
28
29 /**
30 * 1.expireAfterWrite:指定时间内没有创建/覆盖时,会移除该key,下次取的时候触发"同步load"(一个线程执行load)
31 * 2.refreshAfterWrite:指定时间内没有被创建/覆盖,则指定时间过后,再次访问时,会去刷新该缓存,在新值没有到来之前,始终返回旧值
32 * "异步reload"(也是一个线程执行reload)
33 * 3.expireAfterAccess:指定时间内没有读写,会移除该key,下次取的时候从loading中取
34 * 区别:指定时间过后,expire是remove该key,下次访问是同步去获取返回新值;
35 * 而refresh则是指定时间后,不会remove该key,下次访问会触发刷新,新值没有回来时返回旧值
36 *
37 * 同时使用:可避免定时刷新+定时删除下次访问载入
38 */
39 private static final LoadingCache<String, String> cache = CacheBuilder.newBuilder()
40 .maximumSize(1000)
41 //.refreshAfterWrite(1, TimeUnit.SECONDS)
42 .expireAfterWrite(1, TimeUnit.SECONDS)
43 //.expireAfterAccess(1,TimeUnit.SECONDS)
44 .build(new CacheLoader<String, String>() {
45 @Override
46 public String load(String key) throws Exception {
47 System.out.println(Thread.currentThread().getName() +"==load start=="+",时间=" + new Date());
48 // 模拟同步重载耗时2秒
49 Thread.sleep(2000);
50 String value = "load-" + new Random().nextInt(10);
51 System.out.println(
52 Thread.currentThread().getName() + "==load end==同步耗时2秒重载数据-key=" + key + ",value="+value+",时间=" + new Date());
53 return value;
54 }
55
56 @Override
57 public ListenableFuture<String> reload(final String key, final String oldValue)
58 throws Exception {
59 System.out.println(
60 Thread.currentThread().getName() + "==reload ==异步重载-key=" + key + ",时间=" + new Date());
61 return service.submit(new Callable<String>() {
62 @Override
63 public String call() throws Exception {
64 /* 模拟异步重载耗时2秒 */
65 Thread.sleep(2000);
66 String value = "reload-" + new Random().nextInt(10);
67 System.out.println(Thread.currentThread().getName() + "==reload-callable-result="+value+ ",时间=" + new Date());
68 return value;
69 }
70 });
71 }
72 });
73
74 //倒计时器
75 private static CountDownLatch latch = new CountDownLatch(1);
76
77 public static void main(String[] args) throws Exception {
78
79 System.out.println("启动-设置缓存" + ",时间=" + new Date());
80 cache.put("name", "张三");
81 System.out.println("缓存是否存在=" + cache.getIfPresent("name"));
82 //休眠
83 Thread.sleep(2000);
84 //System.out.println("2秒后"+",时间="+new Date());
85 System.out.println("2秒后,缓存是否存在=" + cache.getIfPresent("name"));
86 //启动3个线程
87 for (int i = 0; i < 3; i++) {
88 startThread(i);
89 }
90
91 // -1直接=0,唤醒所有线程读取缓存,模拟并发访问缓存
92 latch.countDown();
93 //模拟串行读缓存
94 Thread.sleep(5000);
95 System.out.println(Thread.currentThread().getName() + "休眠5秒后,读缓存="+cache.get("name")+",时间=" + new Date());
96 Thread.sleep(500);
97 System.out.println(Thread.currentThread().getName() + "距离上一次读0.5秒后,读缓存="+cache.get("name")+",时间=" + new Date());
98 Thread.sleep(2000);
99 System.out.println(Thread.currentThread().getName() + "距离上一次读2秒后,读缓存="+cache.get("name")+",时间=" + new Date());
100 }
101
102 private static void startThread(int id) {
103 Thread t = new Thread(new Runnable() {
104 @Override
105 public void run() {
106 try {
107 System.out.println(Thread.currentThread().getName() + "...begin" + ",时间=" + new Date());
108 //休眠,当倒计时器=0时唤醒线程
109 latch.await();
110 //读缓存
111 System.out.println(
112 Thread.currentThread().getName() + "并发读缓存=" + cache.get("name") + ",时间=" + new Date());
113 } catch (Exception e) {
114 e.printStackTrace();
115 }
116 }
117 });
118
119 t.setName("Thread-" + id);
120 t.start();
121 }
122 }

结果分析

1.expireAfterWrite:当 创建 或 写 之后的 有效期到达时,数据会被自动从缓存中移除

启动-设置缓存,时间=Thu May 17 17:55:36 CST 2018-->主线程启动,缓存创建完毕并设值,即触发写缓存
缓存是否存在=张三
2秒后,缓存是否存在=null--》设定了1秒自动删除缓存,2秒后缓存不存在
Thread-0...begin,时间=Thu May 17 17:55:38 CST 2018--》38秒时,启动3个线程模拟并发读:三个线程读缓存,由于缓存不存在,阻塞在get方法上,等待其中一个线程去同步load数据
Thread-1...begin,时间=Thu May 17 17:55:38 CST 2018
Thread-2...begin,时间=Thu May 17 17:55:38 CST 2018
Thread-1==load start==,时间=Thu May 17 17:55:38 CST 2018---线程1,同步载入数据load()
Thread-1==load end==同步耗时2秒重载数据-key=name,value=load-2,时间=Thu May 17 17:55:40 CST 2018--线程1,同步载入数据load()完毕!,即40秒时写入数据:load-2
Thread-0并发读缓存=load-2,时间=Thu May 17 17:55:40 CST 2018---线程1同步载入数据load()完毕后,3个阻塞在get方法的线程得到缓存值:load-2
Thread-1并发读缓存=load-2,时间=Thu May 17 17:55:40 CST 2018
Thread-2并发读缓存=load-2,时间=Thu May 17 17:55:40 CST 2018
main==load start==,时间=Thu May 17 17:55:43 CST 2018---主线程访问缓存不存在,执行load()
main==load end==同步耗时2秒重载数据-key=name,value=load-4,时间=Thu May 17 17:55:45 CST 2018---load()完毕!45秒时写入数据:load-4
main休眠5秒后,读缓存=load-4,时间=Thu May 17 17:55:45 CST 2018---主线程得到缓存:load-4
main距离上一次读0.5秒后,读缓存=load-4,时间=Thu May 17 17:55:45 CST 2018--距离上一次写才0.5秒,数据有效:load-4
main==load start==,时间=Thu May 17 17:55:47 CST 2018-47秒时,距离上一次写45秒,超过了1秒,数据无效,再次load()
main==load end==同步耗时2秒重载数据-key=name,value=load-8,时间=Thu May 17 17:55:49 CST 2018--49秒时load()完毕:load-8
main距离上一次读2秒后,读缓存=load-8,时间=Thu May 17 17:55:49 CST 2018--打印get的缓存结果:load-8

2.expireAfterAccess:当 创建 或 写 或 读 之后的 有效期到达时,数据会被自动从缓存中移除

修改测试代码98、99行:

Thread.sleep(700);
System.out.println(Thread.currentThread().getName() + "距离上一次读0.5秒后,读缓存="+cache.get("name")+",时间=" + new Date());

启动-设置缓存,时间=Thu May 17 18:32:38 CST 2018
缓存是否存在=张三
2秒后,缓存是否存在=null
Thread-0...begin,时间=Thu May 17 18:32:40 CST 2018
Thread-1...begin,时间=Thu May 17 18:32:40 CST 2018
Thread-2...begin,时间=Thu May 17 18:32:40 CST 2018
Thread-2==load start==,时间=Thu May 17 18:32:40 CST 2018
Thread-2==load end==同步耗时2秒重载数据-key=name,value=load-6,时间=Thu May 17 18:32:42 CST 2018
Thread-0并发读缓存=load-6,时间=Thu May 17 18:32:42 CST 2018
Thread-1并发读缓存=load-6,时间=Thu May 17 18:32:42 CST 2018
Thread-2并发读缓存=load-6,时间=Thu May 17 18:32:42 CST 2018
main==load start==,时间=Thu May 17 18:32:45 CST 2018
main==load end==同步耗时2秒重载数据-key=name,value=load-7,时间=Thu May 17 18:32:47 CST 2018----47秒时写
main休眠5秒后,读缓存=load-7,时间=Thu May 17 18:32:47 CST 2018
main距离上一次读0.5秒后,读缓存=load-7,时间=Thu May 17 18:32:48 CST 2018---48秒读
main距离上一次读0.5秒后,读缓存=load-7,时间=Thu May 17 18:32:49 CST 2018--49秒距离上一次写47秒,间距大于2秒,但是没有触发load() ,因为48秒时又读了一次,刷新了缓存有效期

3.refreshAfterWrite:当 创建 或 写 之后的 有效期到达时,数据会被自动刷新(注意不是删除是刷新)。

启动-设置缓存,时间=Thu May 17 18:39:59 CST 2018--》59秒写
缓存是否存在=张三
main==reload ==异步重载-key=name,时间=Thu May 17 18:40:01 CST 2018--》01秒,2秒后距离上次写超过1秒,reload异步重载
2秒后,缓存是否存在=张三--》距离上一次写过了2秒,但是会立即返回缓存
Thread-0...begin,时间=Thu May 17 18:40:01 CST 2018--》01秒3个线程并发访问
Thread-1...begin,时间=Thu May 17 18:40:01 CST 2018
Thread-2...begin,时间=Thu May 17 18:40:01 CST 2018
Thread-2并发读缓存=张三,时间=Thu May 17 18:40:01 CST 2018--》01秒3个线程都立即得到了缓存
Thread-0并发读缓存=张三,时间=Thu May 17 18:40:01 CST 2018
Thread-1并发读缓存=张三,时间=Thu May 17 18:40:01 CST 2018
pool-1-thread-1==reload-callable-result=reload-5,时间=Thu May 17 18:40:03 CST 2018--》01秒时的异步,2秒后也就是03秒时,查询结果:reload-5
main==reload ==异步重载-key=name,时间=Thu May 17 18:40:06 CST 2018--》06秒时,距离上一次写时间超过1秒,reload异步重载
main休眠5秒后,读缓存=reload-5,时间=Thu May 17 18:40:06 CST 2018--》06秒时,reload异步重载,立即返回旧值reload-5
main距离上一次读0.5秒后,读缓存=reload-5,时间=Thu May 17 18:40:07 CST 2018
main距离上一次读0.5秒后,读缓存=reload-5,时间=Thu May 17 18:40:07 CST 2018
pool-1-thread-2==reload-callable-result=reload-4,时间=Thu May 17 18:40:08 CST 2018--》06秒时的异步重载,2秒后也就是08秒,查询结果:reload-4

回到顶部

三、源码剖析

前面一节简单演示了google cache的几种用法,本节细看源码。

3.1 简介

我们就从构造器CacheBuilder的源码注释,来看一下google cache的简单介绍:

 //LoadingCache加载缓存和缓存实例是以下的特性的组合:
1 A builder of LoadingCache and Cache instances having any combination of the following features:
2 automatic loading of entries into the cache-》把数据实体自动载入到缓存中去-》基本特性
3 least-recently-used eviction when a maximum size is exceeded-》当缓存到达最大数量时回收最少使用的数据-》限制最大内存,避免内存被占满-》高级特性,赞

GuavaCache学习笔记三:底层源码阅读的相关教程结束。

《GuavaCache学习笔记三:底层源码阅读.doc》

下载本文的Word格式文档,以方便收藏与打印。