Review historyHow Blurring-Sharpening Process Models for Collaborative Filtering got from its first submission to SIGIR 2023: 2 submissions. Scores use each venue's own scale (hover a score for the reviewer's confidence; 3→5 = changed after the rebuttal). ← CV of Failures Jul 2022CIKM 2022mean 0, rejected Apr 2023SIGIR 2023mean 1, accepted About the full reviews on this page. Reviews that are already public on OpenReview are only linked. For the other rounds I share the reviews my paper received myself, under three rules: only for papers that were eventually accepted, only conference reviews (never journal reviews), and only as received — reviewers appear under the anonymous IDs or numbers the venue gave them, and I do not know who they are. The venue rules I checked are quoted with each round; none of them prohibits authors from sharing the reviews of their own submission. CIKM 2022 ✗ rejectedsubmitted 2022-05-12-211mean 0
reviews not public — shared by the authors download as .md
What the reviewers raised
CIKM is an ACM conference and does not publish reviews. ACM’s Peer Review Policy FAQ keeps reviewer identities and review content confidential, but states that “authors are permitted to use and share the text of the reviews that they receive”. Full reviewsReviewer 1: rating -2 (reject)Overall evaluation -2 (reject) Overall evaluation (comments)I suggest rejecting this submission for following reasons: Strengths and reasons to accept1. The method is straightforward (perturbation and recovery) and easy to follow. Weaknesses and limitations1. In the ablation study, it is unclear how to set up the correct model structure with respect to the residual connection. It is not clear why the residual connection improves the performance on one dataset instead of the others. Reviewer 2: rating 1 (weak accept)Overall evaluation 1 (weak accept) Overall evaluation (comments)This paper is somewhat interesting as it studies collaborative filtering by Blurring-Sharpening process models. It seems that this is the first study in this regard. The proposed model does not need training ans exhibit good performanced compared to many recent models. Strengths and reasons to accept(1) This paper is the first study that proposes the blurring-sharpening process paradigm for collaborative filtering/ Weaknesses and limitations(1) The motivation of why using Blurring-Sharpening process is not introduced in Sec. 1. In the current version, it looks like there is no existing studies in this regard, so this study tries this technique for recommendation. Reviewer 3: rating 1 (weak accept)Overall evaluation 1 (weak accept) Overall evaluation (comments)In this paper, the authors propose a novel paradigm of Blurring-Sharpening Process Model (BSPM) for collaborative filtering. The proposed blurring and sharpening processes are formulated as differential equations. They also show that some of the existing graph convolution-based collaborative filtering methods are special cases of the proposed model. Strengths and reasons to accept1. design a perturbation-recovery concept, called the Blurring-Sharpening Process Model (BSPM), for collaborative filtering. Weaknesses and limitations1. The code is not release, the reproducibility is unclear. 2. The inference time is much larger than the baselines based on Table 7. It is good that the proposed model has very excitng training time, but with much more inference time. In real-world deployment. People care more about inference time since the training can be performed offline. Meta-review and decisionSIGIR 2023 ✓ acceptedsubmitted 2023-01-23111mean 1
reviews not public — shared by the authors download as .md
What was discussed
SIGIR is an ACM conference and does not publish reviews. ACM’s Peer Review Policy FAQ keeps reviewer identities and review content confidential, but states that “authors are permitted to use and share the text of the reviews that they receive”. Full reviewsReviewer 1: rating 1 (weak accept)Overall recommendation 1 (weak accept) Relevance to SIGIR4 (good) Relevance to SIGIR (comments)Paper is in the full scope of the conference. Originality of the work4 (good) Originality of the work (comments)The idea of the paper is interesting, combining ideas from the vision. Technical soundness4 (good) Technical soundness (comments)The technical soundness of the paper is good, covering most of the relevant aspects. Quality of presentation4 (good) Quality of presentation (comments)The overall presentation is good, the organization of the figures and tables can be improved. Adequacy of citations4 (good) Adequacy of citations (comments)AlI relevant citations are included. Reproducibility of methods2 (poor) Reproducibility of methods (comments)The sorucecode is not available decreasing the reproducibility of the paper. StrengthsThe idea of the paper is interesting, applying concepts from vision for inferring missing interactions. The evaluation section convers reasonable baselines, exploring various aspects of the method on standard datasets. Paper presents both idea and theoretical background behind the method. Overall the paper presents a solid work (including the presentation). WeaknessesDespite only 2 metrics are used, the results are reported only for @k=20. Such large k values do not reflect real-world application and it is not clear whether the improvements are gained on 19th-20th position or on 1st-2nd. This would make a great difference in real application, but form the results it is not visible. The organization of the paper can be improved. The figures and tables are sometimes placed on strange place, which do not contribute to the readability of the paper (e.g., Table 2). The source code is notmade available,which greatly reduces the replicability of the paper and method. Detailed comments to authorsAuthors aim at proposing method for imputing missing interaction in the collaborative filtering setting. The idea of the proposal is interesting, authors suggest using the concept of blurring and in the next phase sharpening of the user item matrix. The paper is generally well written easy to follow (see the shortcoming section), however the source code is not made publicly available, thus greatly reducing the reproducibility of the paper. The method is introduced from both perspectives - the idea and also theoretical background. Authors position the method within the state-of-the-art and in the evaluation section use these as the baselines. The list of baselines covers most of the relevant methods. One of the shortcomings of the evaluation is that authors report metrics only for k=20, which is unrealistic in the real world scenario. Moreover, from the comparison point of view, the method which brings improvement on 20th is better, but this improvement is not so relevant for the real-world application. The evaluation may be skewed by this. To sum it up, the paper presents a solid work, with interesting idea. The method is compared with relevant baselines over standard dataset. I encourage authors to address the shortcomings in the next version of the paper, primarily focusing on the improving the reproducibility of the paper and to report metrics for different k values respectively. Reviewer 2: rating 1 (weak accept)Overall recommendation 1 (weak accept) Relevance to SIGIR5 (excellent) Relevance to SIGIR (comments)none Originality of the work4 (good) Originality of the work (comments)none Technical soundness4 (good) Technical soundness (comments)none Quality of presentation5 (excellent) Quality of presentation (comments)none Adequacy of citations4 (good) Adequacy of citations (comments)none Reproducibility of methods4 (good) Reproducibility of methods (comments)none Strengths1) The authors proposed a Blurring- Sharpening Process Model (BSPM), which can blur the user-item interaction matrix, and sharpen the blurred matrix to derive unknown user-item interactions. It is worth mentioning that this method can provide recommendation results in a very short time without training. Weaknesses1) Although the performance of BSPM can be better than 43 existing CF methods, and the novelty and coverage are also improved. I think the paper lacks the explanation of the theoretical support, such as how tail items are affected during the ODEs solution procedure, resulting in the improvement of evaluation metrics. Rather than the reverse explanation from the results. Detailed comments to authorsOverall, it is hard for me to completely understand the novelty of contributions in different parts of the model. Please refer to my comments of weakness. And the incremental original contribution of the authors isn't clear to me in that context. Anyway, from the experimental results, it performs optimally enough for the recommended scenarios. In addition, the recommendation model without training stage can reduce resource consumption and opens a new era of recommendation systems. Reviewer 3: rating 1 (weak accept)Overall recommendation 1 (weak accept) Relevance to SIGIR4 (good) Relevance to SIGIR (comments)b. This paper focuses on a blurring-sharpening process models for collaborative filtering, which is directly relevant to the topics of SIGIR. Originality of the workSiis wolk takes a fresh perspective on colaborative filtering, which l think is very novel. Technical soundness4 (good) Technical soundness (comments)The model is sound Quality of presentation4 (good) Quality of presentation (comments)The presentation is good and the paper is easy to follow. Adequacy of citations4 (good) Adequacy of citations (comments)Citations are appropriate for the paper. Reproducibility of methods4 (good) Reproducibility of methods (comments)Reproducibility is good. Strengths1.The paper introduces a novel perspective on collaborative filtering by using the Blurring-Sharpening Process model, which is an interesting and innovative approach. 2.The model design in the paper is clear and detailed, providing sufficient information for readers to understand and potentially replicate the study. 3.The experimental results presented in the paper are impressive, as the proposed approach outperforms multiple baseline models on multiple datasets. Weaknesses1.Although the empirical results are promising, the paper lacks sufficient theoretical analysis to explain why the Blurring-Sharpening Process model is effective in improving collaborative filtering. 2.The issue of computational cost is not fully addressed in the paper, particularly in regards to the potential memory usage of real-world user-item interaction matrices. 3.The paper does not appear to address the cold-start problem, which is an important challenge in collaborative filtering. Detailed comments to authorsThis paper proposed a Blurring-Sharpening Process model to address collaborative filtering problem. The empirical results are promising. However, there is still some problems: 1. Lack sufficient theoretical analysis. We are not sure if such good experimental results are supported by theory. 2. The computational cost problem, the user-item interaction matrix in the real world is very large and may result in very large memory consumption. 3. How do you handle cold start problem? Can the same process be applied to new users? Despite some problems, I really appreciate the approach proposed in this paper and I think it provide a very interesting and valuable perspective. Meta-review and decision |