Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(route): Add 微博热搜内容摘要 #13764

Merged
merged 30 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
029c325
fuliba added
iclaud Nov 5, 2023
1aca186
style: auto format
github-actions[bot] Nov 5, 2023
cfc3978
Merge branch 'DIYgod:master' into master
shinemoon Nov 6, 2023
9744ff8
doc refine for fuliba
iclaud Nov 6, 2023
203ce9b
update per lint info
iclaud Nov 6, 2023
cd47847
fix some title and style
iclaud Nov 6, 2023
86cdd2e
Merge branch 'DIYgod:master' into master
shinemoon Nov 7, 2023
26c2d40
update per SSRF concern comment
iclaud Nov 7, 2023
8f0e360
Merge branch 'DIYgod:master' into master
shinemoon Nov 7, 2023
be81fd8
update per SSRF concern during review
iclaud Nov 7, 2023
f42f404
Merge branch 'master' of github.com:shinemoon/RSSHub
iclaud Nov 7, 2023
b99db0c
Update lib/v2/fuliba/radar.js
shinemoon Nov 7, 2023
7222562
Update website/docs/routes/new-media.mdx
shinemoon Nov 7, 2023
ddcd778
turn to wordpress api
iclaud Nov 7, 2023
3d44164
Update website/docs/routes/new-media.mdx
shinemoon Nov 7, 2023
b581080
Merge branch 'DIYgod:master' into master
shinemoon Nov 8, 2023
5253ac1
Merge branch 'DIYgod:master' into master
shinemoon Nov 10, 2023
8fe2496
wip for fulltext weibo hot
iclaud Nov 10, 2023
e128934
refine details and doc
iclaud Nov 11, 2023
a91a352
Merge branch 'DIYgod:master' into master
shinemoon Nov 11, 2023
af4096f
weibo content for hot
iclaud Nov 11, 2023
7157cf9
remove emoji img as not friendly while being handled by some reader tool
iclaud Nov 12, 2023
72ca310
format refinement
iclaud Nov 12, 2023
17358ca
fix single card link address
iclaud Nov 12, 2023
bbef680
update per review comment
iclaud Nov 16, 2023
658582a
doc update
iclaud Nov 16, 2023
b7ce9e9
fetch picture with additional flag
iclaud Nov 16, 2023
6862fdd
minor format adjustment
iclaud Nov 16, 2023
b0158fc
Update doc with refined example
shinemoon Nov 20, 2023
45811a1
get full picture opition as well
iclaud Nov 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/v2/weibo/maintainer.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ module.exports = {
'/group/:gid/:gname?/:routeParams?': ['monologconnor', 'Rongronggg9'],
'/keyword/:keyword/:routeParams?': ['DIYgod', 'Rongronggg9'],
'/oasis/user/:userid': ['kt286'],
'/search/hot': ['xyqfer'],
'/search/hot/:fulltext?': ['xyqfer', 'shinemoon'],
'/super_index/:id/:type?/:routeParams?': ['zengxs', 'Rongronggg9'],
'/timeline/:uid/:feature?/:routeParams?': ['zytomorrow', 'DIYgod', 'Rongronggg9'],
'/user/:uid/:routeParams?': ['DIYgod', 'iplusx', 'Rongronggg9'],
Expand Down
2 changes: 1 addition & 1 deletion lib/v2/weibo/radar.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ module.exports = {
title: '热搜榜',
docs: 'https://docs.rsshub.app/routes/social-media#wei-bo',
source: '/top/summary',
target: '/weibo/search/hot',
target: '/weibo/search/hot/:fulltext?',
},
],
},
Expand Down
2 changes: 1 addition & 1 deletion lib/v2/weibo/router.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ module.exports = (router) => {
router.get('/group/:gid/:gname?/:routeParams?', require('./group'));
router.get('/keyword/:keyword/:routeParams?', require('./keyword'));
router.get('/oasis/user/:userid', require('./oasis/user'));
router.get('/search/hot', require('./search/hot'));
router.get('/search/hot/:fulltext?', require('./search/hot'));
router.get('/super_index/:id/:type?/:routeParams?', require('./super_index'));
router.get('/timeline/:uid/:feature?/:routeParams?', require('./timeline'));
router.get('/user/:uid/:routeParams?', require('./user'));
Expand Down
118 changes: 104 additions & 14 deletions lib/v2/weibo/search/hot.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
const got = require('@/utils/got');
// const weiboUtils = require('../utils');
const config = require('@/config').value;
const { art } = require('@/utils/render');
const cheerio = require('cheerio');
const path = require('path');

module.exports = async (ctx) => {
const {
Expand All @@ -15,21 +18,108 @@ module.exports = async (ctx) => {
},
});

ctx.state.data = {
title: '微博热搜榜',
link: 'https://s.weibo.com/top/summary?cate=realtimehot',
description: '实时热点,每分钟更新一次',
item: data.cards[0].card_group.map((item) => {
if (ctx.params.fulltext === 'fulltext') {
const cardslist = data.cards[0].card_group;
// Topic List
const tlist = cardslist.map((item) => {
const title = item.desc;
const link = `https://m.weibo.cn/search?containerid=100103type%3D1%26q%3D${encodeURIComponent(item.desc)}`;
const description = item.desc;

const plink = `https://m.weibo.cn/api/container/getIndex?containerid=100103type%3D1%26q%3D${encodeURIComponent(item.desc)}`;
const name = item.desc;
return {
title,
description,
link,
title: String(title),
link: String(link),
plink: String(plink),
name: String(name),
shinemoon marked this conversation as resolved.
Show resolved Hide resolved
shinemoon marked this conversation as resolved.
Show resolved Hide resolved
};
}),
};
// ctx.state.data = weiboUtils.sinaimgTvax(ctx.state.data); // no image in the route
});

const resultItems = await Promise.all(
tlist.map((i) =>
ctx.cache.tryGet(i.plink, async () => {
const pInfo = await fetchContent(i.plink);
i.description = pInfo.content;
return i;
})
)
);
ctx.state.data = {
title: '微博热搜榜',
link: 'https://s.weibo.com/top/summary?cate=realtimehot',
description: '实时热点,每分钟更新一次',
shinemoon marked this conversation as resolved.
Show resolved Hide resolved
item: resultItems,
};
} else {
ctx.state.data = {
title: '微博热搜榜',
link: 'https://s.weibo.com/top/summary?cate=realtimehot',
description: '实时热点,每分钟更新一次',
item: data.cards[0].card_group.map((item) => {
const title = item.desc;
const link = `https://m.weibo.cn/search?containerid=100103type%3D1%26q%3D${encodeURIComponent(item.desc)}`;
const description = item.desc;

return {
title,
description,
link,
};
}),
};
}
};

async function fetchContent(url) {
// Fetch the subpageinof
const cookieString = config.weibo.cookies ? config.weibo.cookies : '';
const subres = await got(url, {
headers: {
Cookie: cookieString,
},
});
let demostr = '';
try {
const rdata = JSON.parse(subres.body);
shinemoon marked this conversation as resolved.
Show resolved Hide resolved
const cards = rdata.data.cards;
// Need to find one cards with 'type ==9'
demostr = seekContent(cards);
} catch (e) {
// console.log(e);
// console.log(url);
}
const ret = demostr;
return {
content: ret,
};
}

function seekContent(clist) {
const $ = cheerio.load('<div id="wbcontent"></div>');
const stub = $('#wbcontent');

// Need to find one clist with 'type ==9'
for (let i = 0; i < clist.length; i++) {
const curitem = clist[i];
if (curitem.card_type === 9) {
const curcontent = cheerio.load(curitem.mblog.text);
// curcontent('img').attr('width', '1em').attr('height', '1em');
curcontent('img').remove();
const section = art(path.join(__dirname, 'template/digest.art'), {
author: {
link: curitem.mblog.user.profile_url,
name: curitem.mblog.user.screen_name,
},
msg: curcontent.html(),
link: curitem.scheme,
postinfo: curitem.mblog.created_at,
// picnum: curitem.mblog.pic_num,
// pics: curitem.mblog.pics,
});
stub.append(section);
}
if (curitem.card_type === 11) {
stub.append(seekContent(curitem.card_group));
}
}
shinemoon marked this conversation as resolved.
Show resolved Hide resolved
return stub.html();
}
10 changes: 10 additions & 0 deletions lib/v2/weibo/search/template/digest.art
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<div class="quoted">
<a style="text-decoration: none;" href="{{ author.link }}">{{ author.name }}</a>
<span><a href="{{link }}"> | {{ postinfo }} </a></span>
</div>
<div class="content">
{{@ msg }}
</div>
<div>
------
</div>
shinemoon marked this conversation as resolved.
Show resolved Hide resolved
6 changes: 6 additions & 0 deletions website/docs/routes/social-media.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1704,6 +1704,12 @@ rule

<Route author="xyqfer" example="/weibo/search/hot" path="/weibo/search/hot" anticrawler="1" radar="1" rssbud="1"/>

### 热搜榜(含摘要) {#wei-bo-re-sou-bang-content}

<Route author="shinemoon" example="/weibo/search/hot/fulltext" path="/weibo/search/hot/fulltext" anticrawler="1" radar="1" rssbud="1"/>
shinemoon marked this conversation as resolved.
Show resolved Hide resolved



### 超话 {#wei-bo-chao-hua}

<Route author="zengxs Rongronggg9" example="/weibo/super_index/1008084989d223732bf6f02f75ea30efad58a9/sort_time" path="/weibo/super_index/:id/:type?/:routeParams?" paramsDesc={['超话ID', '类型:见下表', '额外参数;请参阅上面的说明和表格']} anticrawler="1" radar="1" rssbud="1"/>
Expand Down